报告题目：SC2Net: Sparse LSTMs for Sparse Coding
报告人：Dr. Tianyi Zhou (Scientist, Institute of High Performance Computing, A*STAR, Singapore.)
The iterative hard-thresholding algorithm (ISTA) is one of the most popular optimization solvers to achieve sparse codes. However, ISTA suffers from following problems: 1) ISTA employs non-adaptive updating strategy to learn the parameters on each dimension with a fixed learning rate. Such a strategy may lead to inferior performance due to the scarcity of diversity; 2) ISTA does not incorporate the historical information into the updating rules, and the historical information has been proven helpful to speed up the convergence. To address these challenging issues, we propose a novel formulation of ISTA (named as adaptive ISTA) by introducing a novel adaptive momentum vector. To efficiently solve the proposed adaptive ISTA, we recast it as a recurrent neural network unit and show its connection with the well-known long-short-term memory (LSTM) model. With a new proposed unit, we present a neural network (termed SC2Net) to achieve sparse codes in an end-to-end manner. To the best of our knowledge, this is one of the first works to bridge the L1-solver and LSTM, and may provide novel insights in understanding model-based optimization and LSTM. Extensive experiments show the effectiveness of our method on both unsupervised and supervised tasks.
Dr. Zhou Tianyi is a scientist with the Institute of High Performance Computing (IHPC) in Agency for Science, Technology and Research (A*STAR), Singapore. Prior to working in IHPC, he was a senior research engineer with SONY US Research Center in silicon valley and leaded autonomous driving project in SONY. He received the Ph.D. degree in computer science from NTU, Singapore, in 2015. He has published 10+ research papers in renowned venues, including AAAI, IJCAI, CVPR, ECCV, TIP etc. He received the Best Poster Award Honorable Mention at ACML 2012 and Best Paper Award at BeyondLabeler workshop on IJCAI 2016. His current research interests include transfer learning, deep learning and its applications to text classification and computer vision problems.