You are here

Numerical Methods: Design, Analysis, and Computer Implementation of Algorithms

Anne Greenbaum and Timothy P. Chartier
Publisher: 
Princeton University Press
Publication Date: 
2012
Number of Pages: 
454
Format: 
Hardcover
Price: 
95.00
ISBN: 
9780691151229
Category: 
Textbook
[Reviewed by
William J. Satzer
, on
08/6/2012
]

This is a refreshing new take on numerical analysis. It was written for an advanced undergraduate course in the mathematics or computer science curriculum, or in a related discipline. The authors combine a “reasonably rigorous” mathematical approach to elementary numerical analysis with motivation, applications and brief historical notes. They manage to convey a far more integrated sense of numerical analysis than the one that students commonly see. At the same time, they include essentially all the standard topics and more.

The book begins with a chapter on mathematical modeling. This is not one of those introductory chapters full of mostly useless generalities. Instead, the authors take pains to show students where problems in numerical computing arise and to suggest something of the variety of uses for numerical methods. Here there are examples of applications in animation, radiative transport, soccer ball design, fertility in bird populations, and web surfing. Immediately following is a tutorial in MATLAB, the software used throughout the book. (Use of alternative software such as SAGE is possible, but might be inconvenient and distracting since all the code shown in the book is in MATLAB.)

Much of the material is standard stuff. There is the numerical linear algebra (direct and iterative solutions of linear systems and computation of eigenvalues), solution of single nonlinear equations, numerical integration and differentiation, polynomial interpolation, and solutions of ordinary and partial differential equations. Despite the ordinariness of the topics, there is a pleasant freshness here. The writing is lively, the visual presentation is appealing, and there are some great exercises.

Polynomial interpolation is handled fairly peremptorily in comparable texts, but here there is a good though not overly extensive treatment with clear explanations and solid applications. A chapter on Monte Carlo methods — not usually found in texts at this level — is an insightful introduction to a topic of broad interest, also with good exercises. I would have wished for a discussion of singular value decomposition of matrices, also because of its broad usefulness, but the authors do at least discuss LU and QR matrix factorizations. I also particularly liked the treatment of floating point arithmetic and the associated exercises.

An instructor could assemble several different one semester courses using this book — numerical linear algebra and interpolation, or numerical solutions of differential equations — or perhaps a two semester sequence.

This is a charming book, well worth consideration for the next numerical analysis course.


Bill Satzer (wjsatzer@mmm.com) is a senior intellectual property scientist at 3M Company, having previously been a lab manager at 3M for composites and electromagnetic materials. His training is in dynamical systems and particularly celestial mechanics; his current interests are broadly in applied mathematics and the teaching of mathematics.

Preface xiii

Chapter 1: MATHEMATICAL MODELING 1
1.1 Modeling in Computer Animation 2
1.1.1 A Model Robe 2
1.2 Modeling in Physics: Radiation Transport 4
1.3 Modeling in Sports 6
1.4 Ecological Models 8
1.5 Modeling a Web Surfer and Google 11
1.5.1 The Vector Space Model 11
1.5.2 Google's PageRank 13
1.6 Chapter 1 Exercises 14

Chapter 2: BASIC OPERATIONS WITH MATLAB 19
2.1 Launching MATLAB 19
2.2 Vectors 20
2.3 Getting Help 22
2.4 Matrices 23
2.5 Creating and Running .m Files 24
2.6 Comments 25
2.7 Plotting 25
2.8 Creating Your Own Functions 27
2.9 Printing 28
2.10 More Loops and Conditionals 29
2.11 Clearing Variables 31
2.12 Logging Your Session 31
2.13 More Advanced Commands 31
2.14 Chapter 2 Exercises 32

Chapter 3: MONTE CARLO METHODS 41
3.1 A Mathematical Game of Cards 41
3.1.1 The Odds in Texas Holdem 42
3.2 Basic Statistics 46
3.2.1 Discrete Random Variables 48
3.2.2 Continuous Random Variables 51
3.2.3 The Central Limit Theorem 53
3.3 Monte Carlo Integration 56
3.3.1 Buffon's Needle 56
3.3.2 Estimating p 58
3.3.3 Another Example of Monte Carlo Integration 60
3.4 Monte Carlo Simulation of Web Surfing 64
3.5 Chapter 3 Exercises 67

Chapter 4: SOLUTION OF A SINGLE NONLINEAR EQUATION IN ONE UNKNOWN 71
4.1 Bisection 75
4.2 Taylor's Theorem 80
4.3 Newton's Method 83
4.4 Quasi-Newton Methods 89
4.4.1 Avoiding Derivatives 89
4.4.2 Constant Slope Method 89
4.4.3 Secant Method 90
4.5 Analysis of Fixed Point Methods 93
4.6 Fractals, Julia Sets, and Mandelbrot Sets 98
4.7 Chapter 4 Exercises 102

Chapter 5: FLOATING-POINT ARITHMETIC 107
5.1 Costly Disasters Caused by Rounding Errors 108
5.2 Binary Representation and Base 2 Arithmetic 110
5.3 Floating-Point Representation 112
5.4 IEEE Floating-Point Arithmetic 114
5.5 Rounding 116
5.6 Correctly Rounded Floating-Point Operations 118
5.7 Exceptions 119
5.8 Chapter 5 Exercises 120

Chapter 6: CONDITIONING OF PROBLEMS; STABILITY OF ALGORITHMS 124
6.1 Conditioning of Problems 125
6.2 Stability of Algorithms 126
6.3 Chapter 6 Exercises 129

Chapter 7: DIRECT METHODS FOR SOLVING LINEAR SYSTEMS AND LEAST SQUARES PROBLEMS 131
7.1 Review of Matrix Multiplication 132
7.2 Gaussian Elimination 133
7.2.1 Operation Counts 137
7.2.2 LU Factorization 139
7.2.3 Pivoting 141
7.2.4 Banded Matrices and Matrices for Which Pivoting Is Not Required 144
7.2.5 Implementation Considerations for High Performance 148
7.3 Other Methods for Solving Ax = b 151
7.4 Conditioning of Linear Systems 154
7.4.1 Norms 154
7.4.2 Sensitivity of Solutions of Linear Systems 158
7.5 Stability of Gaussian Elimination with Partial Pivoting 164
7.6 Least Squares Problems 166
7.6.1 The Normal Equations 167
7.6.2 QR Decomposition 168
7.6.3 Fitting Polynomials to Data 171
7.7 Chapter 7 Exercises 175

Chapter 8: POLYNOMIAL AND PIECEWISE POLYNOMIAL INTERPOLATION 181
8.1 The Vandermonde System 181
8.2 The Lagrange Form of the Interpolation Polynomial 181
8.3 The Newton Form of the Interpolation Polynomial 185
8.3.1 Divided Differences 187
8.4 The Error in Polynomial Interpolation 190
8.5 Interpolation at Chebyshev Points and chebfun 192
8.6 Piecewise Polynomial Interpolation 197
8.6.1 Piecewise Cubic Hermite Interpolation 200
8.6.2 Cubic Spline Interpolation 201
8.7 Some Applications 204
8.8 Chapter 8 Exercises 206

Chapter 9: NUMERICAL DIFFERENTIATION AND RICHARDSON EXTRAPOLATION 212
9.1 Numerical Differentiation 213
9.2 Richardson Extrapolation 221
9.3 Chapter 9 Exercises 225

Chapter 10: NUMERICAL INTEGRATION 227
10.1 Newton-Cotes Formulas 227
10.2 Formulas Based on Piecewise Polynomial Interpolation 232
10.3 Gauss Quadrature 234
10.3.1 Orthogonal Polynomials 236
10.4 Clenshaw-Curtis Quadrature 240
10.5 Romberg Integration 242
10.6 Periodic Functions and the Euler-Maclaurin Formula 243
10.7 Singularities 247
10.8 Chapter 10 Exercises 248

Chapter 11: NUMERICAL SOLUTION OF THE INITIAL VALUE PROBLEM FOR ORDINARY DIFFERENTIAL EQUATIONS 251
11.1 Existence and Uniqueness of Solutions 253
11.2 One-Step Methods 257
11.2.1 Euler's Method 257
11.2.2 Higher-Order Methods Based on Taylor Series 262
11.2.3 Midpoint Method 262
11.2.4 Methods Based on Quadrature Formulas 264
11.2.5 Classical Fourth-Order Runge-Kutta and Runge-Kutta-Fehlberg Methods 265
11.2.6 An Example Using MATLAB's ODE Solver 267
11.2.7 Analysis of One-Step Methods 270
11.2.8 Practical Implementation Considerations 272
11.2.9 Systems of Equations 274
11.3 Multistep Methods 275
11.3.1 Adams-Bashforth and Adams-Moulton Methods 275
11.3.2 General Linear m-Step Methods 277
11.3.3 Linear Difference Equations 280
11.3.4 The Dahlquist Equivalence Theorem 283
11.4 Stiff Equations 284
11.4.1 Absolute Stability 285
11.4.2 Backward Differentiation Formulas (BDF Methods) 289
11.4.3 Implicit Runge-Kutta (IRK) Methods 290
11.5 Solving Systems of Nonlinear Equations in Implicit Methods 291
11.5.1 Fixed Point Iteration 292
11.5.2 Newton's Method 293
11.6 Chapter 11 Exercises 295

Chapter 12: MORE NUMERICAL LINEAR ALGEBRA: EIGENVALUES AND ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS 300
12.1 Eigenvalue Problems 300
12.1.1 The Power Method for Computing the Largest Eigenpair 310
12.1.2 Inverse Iteration 313
12.1.3 Rayleigh Quotient Iteration 315
12.1.4 The QR Algorithm 316
12.1.5 Google's PageRank 320
12.2 Iterative Methods for Solving Linear Systems 327
12.2.1 Basic Iterative Methods for Solving Linear Systems 327
12.2.2 Simple Iteration 328
12.2.3 Analysis of Convergence 332
12.2.4 The Conjugate Gradient Algorithm 336
12.2.5 Methods for Nonsymmetric Linear Systems 334
12.3 Chapter 12 Exercises 345

Chapter 13: NUMERICAL SOLUTION OF TWO-POINT BOUNDARY VALUE PROBLEMS 350
13.1 An Application: Steady-State Temperature Distribution 350
13.2 Finite Difference Methods 352
13.2.1 Accuracy 354
13.2.2 More General Equations and Boundary Conditions 360
13.3 Finite Element Methods 365
13.3.1 Accuracy 372
13.4 Spectral Methods 374
13.5 Chapter 13 Exercises 376

Chapter 14: NUMERICAL SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS 379
14.1 Elliptic Equations 381
14.1.1 Finite Difference Methods 381
14.1.2 Finite Element Methods 386
14.2 Parabolic Equations 388
14.2.1 Semidiscretization and the Method of Lines 389
14.2.2 Discretization in Time 389
14.3 Separation of Variables 396
14.3.1 Separation of Variables for Difference Equations 400
14.4 Hyperbolic Equations 402
14.4.1 Characteristics 402
14.4.2 Systems of Hyperbolic Equations 403
14.4.3 Boundary Conditions 404
14.4.4 Finite Difference Methods 404
14.5 Fast Methods for Poisson's Equation 409
14.5.1 The Fast Fourier Transform 411
14.6 Multigrid Methods 414
14.7 Chapter 14 Exercises 418

APPENDIX A REVIEW OF LINEAR ALGEBRA 421
A.1 Vectors and Vector Spaces 421
A.2 Linear Independence and Dependence 422
A.3 Span of a Set of Vectors; Bases and Coordinates; Dimension of a Vector Space 423
A.4 The Dot Product; Orthogonal and Orthonormal Sets; the Gram-Schmidt Algorithm 423
A.5 Matrices and Linear Equations 425
A.6 Existence and Uniqueness of Solutions; the Inverse; Conditions for Invertibility 427
A.7 Linear Transformations; the Matrix of a Linear Transformation 431
A.8 Similarity Transformations; Eigenvalues and Eigenvectors 432
APPENDIX B TAYLOR'S THEOREM IN MULTIDIMENSIONS 436

References 439
Index 445