20181126 王闯：Understanding stochastic gradient methods for high-dimensional inference: Dynamics, Guarantees, and Optimality
报告时间：2018年11月26日 16：00-17：00
报告地点：明德主楼1016会议室
报告题目：Understanding stochastic gradient methods
for
high-dimensional inference: Dynamics, Guarantees, and Optimality
报告摘要：
Inference and
learning from high-dimensional data are at the heart of modern signal processing
and machine learning. Stochastic gradient-based algorithms have achieved
surprisingly good performance for many convex and non-convex problems, as
they are fast and memory-efficient. However, their empirical success highly
depends on the careful choices of hyper-parameters, such as the
learning rates. A deep understanding of why and when stochastic gradient
methods work or do not work is still not quite clear, and strong theoretical
guarantees are also limited.
In this talk, I will present a rigorous framework for analyzing the exact
dynamics of these algorithms in the high-dimensional limit with applications
ranging from nonlinear regression to more challenging tasks such as
subspace tracking using partially observed data, independent component
analysis, and the training of generative adversarial networks. Leveraging tools
from statistical physics and high-dimensional probability theory, we show
that one can use a few ordinary differential equations or partial differential
equations to precisely characterize the limiting dynamics associated with
these stochastic gradient methods.
Based on this analysis, we provide a useful insight that in the
high-dimensional limit, the original coupled dynamics associated with the
algorithms will be asymptotically “decoupled”, with each coordinate
independently solving a 1-D effective minimization problem via stochastic
gradient descent. Exploiting this insight to design new algorithms for
achieving optimal trade-offs between computational and statistical efficiency
proves an interesting line of research.
报告人简历：
Chuang Wang received his Ph.D. degree in Theoretical Physics from the Institute
of Theoretical Physics, Chinese Academy of Science, Beijing, China, in 2015. He
then joined the Paulson School of Engineering and Applied Sciences at
Harvard University, first as a Postdoctoral Fellow (Feb 2015 - Jan 2018) and
more recently as a Research Associate (Feb 2018 - now) in the Signals,
Information, and Networks Group. His research interests include theoretical
aspects of high-dimensional signal and information processing; machine
learning; imaging and imaging processing; probabilistic graphical models,
physics-inspired optimization algorithms. He won the Best Student Paper
Award at the IEEE GlobalSip Conference in 2014.
