11.26 | Towards Relation Inference and Causal Reasoning in Video

演讲者 Prof Tat-Seng CHUA 头衔职位 National University of Singapore, School of Computing 时间 2019 年 11 月 26 号（周二）下午 3 点 地点 张江校区软件楼 105 IBM 会议室 联系人 李析燃，lixiran@fudan.edu.cn

演讲简介

Recent advances in personal video recording devices and video sharing platforms have resulted in an explosive growth of online videos. This has given rise to demands for various video content understanding tasks, including video search, question answering (QA), and most importantly, content filtering. It motivates us to research into comprehensive and fine-grained video content analytics. A key limitation of current methods is that they rely on generalized visual features without any semantics. They ignore a key aspect of content, which is the relations among objects and between objects and other types of entities such as the events. The current approaches are therefore unable to capture the subtle and fine-grained interactions and semantics within the video.

In this research, we focus on extending the research from visual object recognition to that of relation inference. It permits the system to extract basic knowledge relation triplets in the form of which elevates the extractable semantic of video to that of text, and supports a wide range of higher level video-text fusion tasks such as the multi-perspective video captioning, video QA, and multimodal knowledge graph. It also supports causal reasoning, such as the counterfactual inference. In this talk, I will present the background on object recognition, and present our recent works in transforming the research from object recognition to video relation inference and causal reasoning. I will conclude the talk by outlining several future research directions and applications. As part of this research, we have constructed and released a large-scale video dataset to facilitate relevant research in this emerging field. The dataset has been used for a grand challenge in the ACM Multimedia 2019 conference.

关于讲者

Dr Chua is the KITHCT Chair Professor at the School of Computing, National University of Singapore (NUS). He is also the Distinguished Visiting Professor of Tsinghua University. Dr Chua was the Founding Dean of the School of Computing from 1998-2000. His main research interests include unstructured data analytics, multimedia information retrieval, recommendation and conversation systems, and the emerging applications in E-commerce, wellness and Fintech. Dr Chua is the co-Director of NExT, a joint research Center between NUS and Tsinghua, focusing on Extreme Search.

Dr Chua is the recipient of the 2015 ACM SIGMM Achievements Award for the Outstanding Technical Contributions to Multimedia Computing, Communications and Applications. He is the Chair of steering committee of ACM International Conference on Multimedia Retrieval (ICMR) and Multimedia Modeling (MMM) conference series. He was the General Co-Chair of ACM Multimedia 2005, ACM CIVR (now ACM ICMR) 2005, ACM SIGIR 2008, and ACM Web Science 2015. He serves in the editorial boards of three international journals. Dr. Chua is the co-Founder of two technology startup companies in Singapore. He holds a PhD from the University of Leeds, UK.