Date&Time : 

2021 / 05 / 28 (Fri) 14:20

Speaker : 

 許志仲教授 (Prof. Chih-Chung Hsu)

國立成功大學數據科學研究所 (Inst. of Data Science, NCKU)

Topic : 

DeepFake Detection: Trend and challenge

Abstract : 

A GAN-based technique named DeepFake has brought many adverse effects recently. To effectively recognize the DeepFake videos, deep neural networks are usually adopted based on a pre-collected and large-scale training set to learn the discriminative feature representation for fake videos. In real-world scenarios, it is hard to collect a large-scale, well-annotated, and high-quality training set of malicious DeepFake videos generated by a brand-new type of GANs. Furthermore, it might be unknown of the methodology of such new GANs, implying that it is hard to collect their training samples. Therefore, the performance of the existing DeepFake detectors might be restricted for real-world applications. In this talk, a novel low-shot DeepFake video detection scheme is proposed to tackle this issue. First, we explore the universal and intrinsic features of fake videos in a self-supervised learning fashion based on the large-scale unlabeled dataset. The two data augmentation policies are performed on the facial image sequences to obtain their weakly and strongly augmented facial image sequences pair. Furthermore, a novel Spatio-temporal Convolutional Transformer (SCT) is also proposed to effectively capture both temporal and spatial features of DeepFake videos, and followed by updating our SCT weights based on normalized temperature-scaled cross-entropy (NT-Xent) loss. In this way, the intrinsic features of such DeepFake videos should learn in a self-supervised manner without label information. Finally, a simple classifier is adopted to recognize the features obtained by our SCT to judge whether the input video is fake or real. Comprehensive experimental results also demonstrated that the proposed method achieves superior performance compared to other state-of-the-art fake face detectors of the limited labeling data scenarios.

Bio :

Dr. Hsu received his B.S. degree in Information management from Ling-Tung University of Science and Technology, Taiwan, in 2004, and the M.S. and Ph.D. degrees in Electrical Engineering from National Yunlin University of Science and Technology and National Tsing-Hua University (NTHU), Taiwan, in 2007 and 2014, respectively. Dr. Hsu was a postdoctoral researcher with the Institute of Communications Engineering, NTHU, from 2014 to 2017. He is currently an Assistant Professor with the Institute of Data Science, National Cheng Kung University (NCKU), Taiwan. Before joining NCKU, He was an Assistant Professor with the Department of Management Information Systems, National Pingtung University of Science and Technology as an assistant professor since February 2018. His research interests mainly lie in computer vision and machine/deep learning with applications to image and video processing. Dr. Hsu is Senior Member of Institute of Electrical and Electronics Engineers (IEEE) since Oct. 2020. He received the first-place award of ACM Multimedia Social Media Prediction Challenge in 2017 and 2019, and a top 10\% paper award from IEEE International Workshop on Multimedia Signal Processing (MMSP) 2013. In 2019, Dr. Hsu received the best student paper award from IEEE International Conference on Image Processing (ICIP) in 2019. He also won the 3rd place award of Learning to Drive Challenge from IEEE International Conference on Computer Vision (ICCV) and has been invited to give a talk in it. Dr. Hsu received the 3rd place award of Visual Inductive Priors for Data-Efficient Computer Vision Challenge from European Conference on Computer Vision (ECCV).