The Class of ’27 Lecture Series is a special lecture held each year. It was established in 1960 to honor the first chair of the Math Sciences Department, Professor Edwin Allen. The three members of the class of 1927 who established this series are Issac Arnold, Alexander Hassan, and Isadore Fixman.
Coordinate descent is an approach for minimizing functions in which only a subset of the variables are allowed to change at each iteration, while the remaining variables are held fixed. This approach has been popular in applications since the earliest days of optimization, because it is intuitive and because the low-dimensional searches that take place at each iteration are inexpensive in many applications. In recent years, the popularity of coordinate descent methods has grown further because of their usefulness in data analysis. In this talk we describe situations in which coordinate descent methods are useful, and discuss several variants of these methods and their convergence properties. We describe recent analysis of the convergence of asynchronous parallel versions of these methods, which achieve high efficiency on multicore computers.
Optimization formulations and algorithms are vital tools for solving problems in data analysis. There has been particular interest in some fundamental, elementary, optimization algorithms that were previously thought to have only niche appeal. Stochastic gradient, coordinate descent, and accelerated first-order methods are three examples. We outline applications in which these approaches are useful, discuss the basic properties of these methods, and survey some recent developments in the analysis of their convergence behavior.