This text aims to give a through description of some of the most commonly used techniques and algorithms of numerical linear algebra and optimization accompanied by rigorous mathematical analysis. Its prerequisites effectively include introductory courses in linear algebra and advanced calculus. Much of the book would be accessible to a good undergraduate student, though the latter parts present results in the more general settings of Hilbert or Banach spaces and require some functional analysis. The book is split roughly half and half between numerical linear algebra and optimization.
Although books such as this one often appeal most to practically oriented readers (applied mathematicians, physicists, engineers, economists), the author wants to develop in that reader an interest in the underlying mathematical analysis as well. Nonetheless, most readers approaching this book would expect to gain a basic knowledge of numerical methods for matrix calculations and optimization and how they are implemented.
When I once worked for a supercomputer manufacturer as a “computational scientist”, I learned pretty quickly that an enormous amount of supercomputer time was devoted to solving systems of linear equations. Naturally enough, there was a significant focus on using the fastest algorithms to do this. Yet few people involved in development and support knew much of anything about linear algebra. This often led to a blind reliance on canned software. Much of this software is quite good, but only if used appropriately. I could tell you stories… If it does nothing more, a book like the current one should guide a reader to identify appropriate algorithms and find or develop appropriate software to implement them.
The book begins with a quick review of linear algebra. This includes some concepts, like matrix norms, that may be new material to some readers. A short chapter follows on general principles of numerical analysis that apply to matrix computations.
The author devotes a long chapter to describe sources of problems in numerical matrix analysis. Here he includes applications arising from finite-difference methods for various boundary value problems and eigenvalue problems as well as interpolation and approximation. The objective here seems to be partly motivation and partly a desire to show the chain of analysis that goes from the appropriate analytical formulation (e.g., a differential equation) to a discrete linear algebra problem. The remaining chapters on numerical linear algebra focus on direct and iterative methods for solving linear systems and methods for computing eigenvalues.
The chapters on optimization address both continuous and discrete problems. On the continuous side, the author describes the usual gradient, relaxation and conjugate gradient methods for both constrained and unconstrained optimization. The book concludes with chapters on linear and nonlinear programming.
The combination of numerical linear algebra with optimization in this text is curious, not because the topics are at odds, but because they are relatively independent. At the same time, both parts of the book would have benefited for a more extended treatment. The linear algebra, in particular, seems rather thin. It leaves out important topics like singular value decomposition, it is short on examples, and it pays minimal attention to questions that arise in ill-conditioned systems.
Bill Satzer (email@example.com) is a senior intellectual property scientist at 3M Company, having previously been a lab manager at 3M for composites and electromagnetic materials. His training is in dynamical systems and particularly celestial mechanics; his current interests are broadly in applied mathematics and the teaching of mathematics.
Preface; Part I. Numerical Linear Algebra: 1. A summary of results on matrices; 2. General results in the numerical analysis of matrices; 3. Sources of problems in the numerical analysis of matrices; 4. Direct methods for the solution of linear systems; 5. Iterative methods for the solution of linear systems; 6. Methods for the calculation of eigenvalues and eigenvectors; Part II. Optimisation: 7. A review of differential calculus. Some applications; 8. General results on optimisation. Some algorithms; 9. Introduction to non-linear programming; 10. Linear programming; Bibliography and comments; Main notations used; Index.