Selected Applications of Convex Optimization

There are many methods from applied statistics and machine learning in common use. They can seem like an ad hoc bag of tricks with little organization. In part, this is because they are an ad hoc bag of tricks! But only in part.

Data science methods usually boil down to solving an optimization problem. Thinking about a method in terms of its underlying optimization problem helps understand how the method works, suggests how to implement the method numerically, and may show how the method is related to other methods. Two methods may have different names, and be popular in different communities, and yet only differ by a small change in objective function.

The optimization problems alluded to above are often convex. This is fortunate because, very roughly speaking, optimization problems are tractable if and only if they are convex. There are tractable non-convex optimization problems and intractable convex problems, but convex problems have a systematic theory and quality software implementations.

Li Li’s book Selected Applications of Convex Optimization could be seen as a supplement to the popular book Convex Optimization by Stephen Boyd and Lieven Vandenberghe. Li says in the preface that his choice of material was “significantly influenced” by Boyd and Vandenberghe’s book and that their book is required reading for his students. While many of the topics in Li’s book are in Boyd and Vandenberghe, these topics are organized differently here and covered in more detail.

Selected Applications of Convex Optimization is a brief book, only 140 pages, and includes exercises with each chapter. It would be a good supplemental text for an optimization or machine learning course. MATLAB scripts associated with the book are available as extra material on the publisher’s web site.

John D. Cook is an statistical consultant and blogs regularly at The Endeavour.