当前位置: 学院首页 > 学术科研 > 学术看板 > 正文

学术报告——Recent Advances in Zero-Example Video Retrieval

日期:2018-11-13 来源: 作者: 浏览:

报告题目: Recent Advances in Zero-Example Video Retrieval






Zero-example video retrieval is a challenging problem in the multimedia field. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described in natural language text with no visual example provided. The majority of existing methods are concept based, extracting relevant concepts from queries and videos and accordingly establishing associations between the two modalities. In this talk we introduce two recently developed concept-free methods. The first method is Word2VisualVec++, a super version of Word2VisualVec. For a given sentence, Word2VisualVec projects it into a deep video feature space by first vectorizing the sentence by a multi-scale encoding strategy. The encoding result then goes through a multilayer perceptron (MLP) to produce a feature vector. The network is trained such that the Mean Square Error (MSE) between the feature vector of the sentence and the feature vector of a video the sentence is describing is minimized. Word2VisualVec++ improves over Word2VisualVec by substituting an improved marginal ranking loss for the MSE loss. The second method is a dual dense encoding network. The network encodes an input, let it be a video or a natural language sentence, in a dense manner. In particular, by jointly exploiting multi-level encodings including mean pooling, GRU and GRU-CNN, the network models explicitly global, local and temporal patterns in both videos and sentences. As such, a given video / query is encoded into a powerful representation of its own. The effectiveness of the two methods is verified by the very recent NIST TRECVID challenge, by winning both the TRECVID 2018 Ad-hoc Video Search (AVS) task and Video-to-Text (VTT) Matching and Ranking task.


李锡荣,中国人民大学副教授、博士生导师。分别于2005年、2007年获清华大学计算机专业本科、硕士学位,2012年获荷兰阿姆斯特丹大学计算机博士学位。同年5月份加入中国人民大学数据工程与知识工程教育部重点实验室,任讲师。2016年晋升副教授,2017年晋升博导,并入选中国人民大学首批杰出学者支持计划。主要研究领域是人工智能与媒体计算。在相关领域主要国际会议和期刊发表论文 50 余篇,Google scholar被引用数 2000 多次,H指数21。获 ACM MM 2018 Hulu Challenge优胜奖、2017中国多媒体大会优秀论文奖, ACM MM 2016 Grand Challenge AwardPCM 2016 Best Paper Runner-up, ACM SIGMM 2013 Best PhD Thesis AwardIEEE Transactions on Multimedia 2012 Best Paper AwardCIVR 2010 Best Paper Award。曾任ACM MM 2018ICPR 2016 Area Chair。中国计算机学会多媒体专委会委员。







下一条:学术报告——Normalization of Duplicate Records from Multiple Sources