Nonlinear Least Squares for Inverse Problems: Theoretical Foundations and Step-by-Step Guide for Applications

In an inverse problem, we are given a function that maps some unknown parameters to observable data. For example, this function might map the parameters of a partial differential equation boundary value problem into the solution of the boundary value problem. The inverse problem is to invert the mapping and determine the parameters that led to a set of observed data. In the output least squares (OLS) approach, we solve the inverse problem by adjusting the input parameters to minimize the norm of the difference between the output of the forward map and the observed data. This book is a research monograph, largely based on the author’s own published research, that provides both a practical guide to solving nonlinear inverse problems using the OLS approach and a theoretical analysis of conditions under which the OLS problem is well posed or can be made well posed by regularization.

In general, the data and parameters can live in infinite dimensional Hilbert spaces. However, in computational practice it is necessary to parameterize the problem, reducing the problem data and unknown parameters to finite dimensional vectors. This parameterization must be performed with care because an overly coarse parameterization can lead to a loss of resolution in the solution, while an overly fine parameterization can make the OLS problem ill-posed. After the parameterization has been completed, there are many algorithms for the nonlinear least squares problem including the Gauss-Jordan, Levenberg-Marquardt, and quasi-Newton methods. All of these methods depend on the computation of derivatives. The adjoint equation method is an important tool that makes possible the computation of derivatives for very large scale problems.

In addition to these practical computational issues, the well-posedness of the OLS problem is of critical concern. The OLS problem should be well posed in the sense that the solution to the problem is unique and that the solution is stable under small perturbations to the problem data. Since the algorithms used to solved the parameterized nonlinear least squares problem find local minima, it is also important in practice to avoid situations in which the optimization problem has parasitic local minima. A problem is quadratically well posed or Q-wellposed if it is well posed and has only a single global minimum that can be found by local search algorithms. Chavent gives conditions under which the OLS problem will be Q-wellposed. In cases where the OLS problem is not Q-wellposed various regularization methods can be used to alter the problem to make is Q-wellposed. Chavent considers regularization by the choice of parameterization, by Tikhonov regularization, and by smoothing of the observed data.

The book is organized so that readers interested in the more practical aspects can easily dip into the appropriate chapters of the book without having to work through the more theoretical details. The book begins with a chapter that establishes notation and gives several specific examples of inverse problems that are used throughout the book. The computation of derivatives and parameterization techniques are discussed in Chapters 2 and 3. Conditions for Q-wellposedness of the OLS problem are given in Chapter 4. Tikhonov regularization and regularization by smoothing the problem data are discussed in Chapter 5. The theoretical background on quasi-convex and strictly quasi-convex sets and results on Q-wellposedness is presented in chapters 6 through 8.

This book is recommended for readers who are interested in applying the OLS approach to nonlinear inverse problems. These readers should find the material on parameterization, the adjoint equation method, and regularization techniques particularly valuable. This material is relatively accessible even to readers without a very strong background in analysis. The book will also be of interest to readers who want to learn more about the theoretical background and results on quasi-convex sets and Q-wellposedness.

Brian Borchers is a professor of Mathematics at the New Mexico Institute of Mining and Technology. His interests are in optimization and applications of optimization in parameter estimation and inverse problems.