Seminar on Optimization Algorithms

Abstract

This seminar provides a comprehensive overview of optimization algorithms, covering both theoretical foundations and practical applications. We begin with advanced convex programming techniques, including the Alternating Direction Method with Gaussian Back Substitution and optimally linearized ADMM. The seminar then explores modern optimization methods such as Adam and generalized primal–dual hybrid gradient approaches for saddle point problems. We also discuss communication-efficient statistical estimation and the balanced augmented Lagrangian method. Further topics include the global linear convergence of ADMM with multi-block variables, decentralized consensus optimization via the EXTRA algorithm, and stochastic gradient descent with variance reduction. Finally, we examine the role of optimization in double descent and investigate why overparameterized neural networks can still generalize effectively.

Time and Location

Time: 2:00pm-4:00pm, Thursday, 2024 Spring

Location: USTC the 5th Teaching Building and Online

Learning Materials

The materials are stored in Baidu Cloud Drive: PPT.