Topic:
Continual lifelong learning of deep models

Speaker:陳祝嵩 博士(中央研究院資訊科學研究所)

Language: English

Date: 2019/12/20

Time: 14:20~16:20

Location: Delta building R216, NTHU

Abstract:
In this talk, I will review continual lifelong learning of deep models at first. Then, I introduce an approach leveraging the principles of deep model compression with weight pruning, critical weights selection, and progressive networks expansion. By enforcing their integration in an iterative manner, an incremental learning method that is scalable to the number of sequential tasks in a continual learning process is proposed. This approach, dubbed compacting picking and growing (CPG), owns several favorable characteristics. First, it can avoid forgetting (i.e., learn new tasks while remembering all previous tasks). Second, it allows model expansion but can maintain the model compactness when handling sequential tasks. Besides, we show that the knowledge accumulated through learning previous tasks is helpful to adapt to a better model for the new tasks compared to training the models independently with tasks. Experimental results show that the approach can incrementally learn a deep model to tackle multiple tasks without forgetting, while the model compactness is maintained with the performance more satisfiable than individual task training.

Biography:
Dr. Chu-Song Chen is a research fellow/professor of the Institute of Information Science (IIS), Academia Sinica, and an adjunct professor of the Graduate Institute of Networking and Multimedia (GINM), National Taiwan University. His research interests include deep learning, pattern recognition, computer vision, and multimedia. Currently, he serves as an associate editor of the journals Pattern Recognition (Elsevier) and Machine Vision & Applications (Springer). Dr. Chen devotes to deep learning researches since 2014. In this field, he has several publications on top conferences (such as CVPR, ICCV, ACM MM, IJCAI, NeurIPS) and journals (IEEE TPAMI, TNNLS). His works of deep learning of binary features for efficient retrieval (CVPRW15, TPAMI18) are pioneer studies on hash function learning via deep networks, which have been cited more than 400 times on google scholar. His team won the champion of National Intelligent-Manufacture Big-Data (IMBD) analysis challenge in 2019. His recent studies focus on merging and lifelong-learning of deep models, medical image analysis, AI in emergency medicine, as well as 3D environment exploration and reconstruction.