Preface xiii

Chapter 1: MATHEMATICAL MODELING 1

1.1 Modeling in Computer Animation 2

1.1.1 A Model Robe 2

1.2 Modeling in Physics: Radiation Transport 4

1.3 Modeling in Sports 6

1.4 Ecological Models 8

1.5 Modeling a Web Surfer and Google 11

1.5.1 The Vector Space Model 11

1.5.2 Google's PageRank 13

1.6 Chapter 1 Exercises 14

Chapter 2: BASIC OPERATIONS WITH MATLAB 19

2.1 Launching MATLAB 19

2.2 Vectors 20

2.3 Getting Help 22

2.4 Matrices 23

2.5 Creating and Running .m Files 24

2.6 Comments 25

2.7 Plotting 25

2.8 Creating Your Own Functions 27

2.9 Printing 28

2.10 More Loops and Conditionals 29

2.11 Clearing Variables 31

2.12 Logging Your Session 31

2.13 More Advanced Commands 31

2.14 Chapter 2 Exercises 32

Chapter 3: MONTE CARLO METHODS 41

3.1 A Mathematical Game of Cards 41

3.1.1 The Odds in Texas Holdem 42

3.2 Basic Statistics 46

3.2.1 Discrete Random Variables 48

3.2.2 Continuous Random Variables 51

3.2.3 The Central Limit Theorem 53

3.3 Monte Carlo Integration 56

3.3.1 Buffon's Needle 56

3.3.2 Estimating p 58

3.3.3 Another Example of Monte Carlo Integration 60

3.4 Monte Carlo Simulation of Web Surfing 64

3.5 Chapter 3 Exercises 67

Chapter 4: SOLUTION OF A SINGLE NONLINEAR EQUATION IN ONE UNKNOWN 71

4.1 Bisection 75

4.2 Taylor's Theorem 80

4.3 Newton's Method 83

4.4 Quasi-Newton Methods 89

4.4.1 Avoiding Derivatives 89

4.4.2 Constant Slope Method 89

4.4.3 Secant Method 90

4.5 Analysis of Fixed Point Methods 93

4.6 Fractals, Julia Sets, and Mandelbrot Sets 98

4.7 Chapter 4 Exercises 102

Chapter 5: FLOATING-POINT ARITHMETIC 107

5.1 Costly Disasters Caused by Rounding Errors 108

5.2 Binary Representation and Base 2 Arithmetic 110

5.3 Floating-Point Representation 112

5.4 IEEE Floating-Point Arithmetic 114

5.5 Rounding 116

5.6 Correctly Rounded Floating-Point Operations 118

5.7 Exceptions 119

5.8 Chapter 5 Exercises 120

Chapter 6: CONDITIONING OF PROBLEMS; STABILITY OF ALGORITHMS 124

6.1 Conditioning of Problems 125

6.2 Stability of Algorithms 126

6.3 Chapter 6 Exercises 129

Chapter 7: DIRECT METHODS FOR SOLVING LINEAR SYSTEMS AND LEAST SQUARES PROBLEMS 131

7.1 Review of Matrix Multiplication 132

7.2 Gaussian Elimination 133

7.2.1 Operation Counts 137

7.2.2 LU Factorization 139

7.2.3 Pivoting 141

7.2.4 Banded Matrices and Matrices for Which Pivoting Is Not Required 144

7.2.5 Implementation Considerations for High Performance 148

7.3 Other Methods for Solving Ax = b 151

7.4 Conditioning of Linear Systems 154

7.4.1 Norms 154

7.4.2 Sensitivity of Solutions of Linear Systems 158

7.5 Stability of Gaussian Elimination with Partial Pivoting 164

7.6 Least Squares Problems 166

7.6.1 The Normal Equations 167

7.6.2 QR Decomposition 168

7.6.3 Fitting Polynomials to Data 171

7.7 Chapter 7 Exercises 175

Chapter 8: POLYNOMIAL AND PIECEWISE POLYNOMIAL INTERPOLATION 181

8.1 The Vandermonde System 181

8.2 The Lagrange Form of the Interpolation Polynomial 181

8.3 The Newton Form of the Interpolation Polynomial 185

8.3.1 Divided Differences 187

8.4 The Error in Polynomial Interpolation 190

8.5 Interpolation at Chebyshev Points and chebfun 192

8.6 Piecewise Polynomial Interpolation 197

8.6.1 Piecewise Cubic Hermite Interpolation 200

8.6.2 Cubic Spline Interpolation 201

8.7 Some Applications 204

8.8 Chapter 8 Exercises 206

Chapter 9: NUMERICAL DIFFERENTIATION AND RICHARDSON EXTRAPOLATION 212

9.1 Numerical Differentiation 213

9.2 Richardson Extrapolation 221

9.3 Chapter 9 Exercises 225

Chapter 10: NUMERICAL INTEGRATION 227

10.1 Newton-Cotes Formulas 227

10.2 Formulas Based on Piecewise Polynomial Interpolation 232

10.3 Gauss Quadrature 234

10.3.1 Orthogonal Polynomials 236

10.4 Clenshaw-Curtis Quadrature 240

10.5 Romberg Integration 242

10.6 Periodic Functions and the Euler-Maclaurin Formula 243

10.7 Singularities 247

10.8 Chapter 10 Exercises 248

Chapter 11: NUMERICAL SOLUTION OF THE INITIAL VALUE PROBLEM FOR ORDINARY DIFFERENTIAL EQUATIONS 251

11.1 Existence and Uniqueness of Solutions 253

11.2 One-Step Methods 257

11.2.1 Euler's Method 257

11.2.2 Higher-Order Methods Based on Taylor Series 262

11.2.3 Midpoint Method 262

11.2.4 Methods Based on Quadrature Formulas 264

11.2.5 Classical Fourth-Order Runge-Kutta and Runge-Kutta-Fehlberg Methods 265

11.2.6 An Example Using MATLAB's ODE Solver 267

11.2.7 Analysis of One-Step Methods 270

11.2.8 Practical Implementation Considerations 272

11.2.9 Systems of Equations 274

11.3 Multistep Methods 275

11.3.1 Adams-Bashforth and Adams-Moulton Methods 275

11.3.2 General Linear m-Step Methods 277

11.3.3 Linear Difference Equations 280

11.3.4 The Dahlquist Equivalence Theorem 283

11.4 Stiff Equations 284

11.4.1 Absolute Stability 285

11.4.2 Backward Differentiation Formulas (BDF Methods) 289

11.4.3 Implicit Runge-Kutta (IRK) Methods 290

11.5 Solving Systems of Nonlinear Equations in Implicit Methods 291

11.5.1 Fixed Point Iteration 292

11.5.2 Newton's Method 293

11.6 Chapter 11 Exercises 295

Chapter 12: MORE NUMERICAL LINEAR ALGEBRA: EIGENVALUES AND ITERATIVE METHODS FOR SOLVING LINEAR SYSTEMS 300

12.1 Eigenvalue Problems 300

12.1.1 The Power Method for Computing the Largest Eigenpair 310

12.1.2 Inverse Iteration 313

12.1.3 Rayleigh Quotient Iteration 315

12.1.4 The QR Algorithm 316

12.1.5 Google's PageRank 320

12.2 Iterative Methods for Solving Linear Systems 327

12.2.1 Basic Iterative Methods for Solving Linear Systems 327

12.2.2 Simple Iteration 328

12.2.3 Analysis of Convergence 332

12.2.4 The Conjugate Gradient Algorithm 336

12.2.5 Methods for Nonsymmetric Linear Systems 334

12.3 Chapter 12 Exercises 345

Chapter 13: NUMERICAL SOLUTION OF TWO-POINT BOUNDARY VALUE PROBLEMS 350

13.1 An Application: Steady-State Temperature Distribution 350

13.2 Finite Difference Methods 352

13.2.1 Accuracy 354

13.2.2 More General Equations and Boundary Conditions 360

13.3 Finite Element Methods 365

13.3.1 Accuracy 372

13.4 Spectral Methods 374

13.5 Chapter 13 Exercises 376

Chapter 14: NUMERICAL SOLUTION OF PARTIAL DIFFERENTIAL EQUATIONS 379

14.1 Elliptic Equations 381

14.1.1 Finite Difference Methods 381

14.1.2 Finite Element Methods 386

14.2 Parabolic Equations 388

14.2.1 Semidiscretization and the Method of Lines 389

14.2.2 Discretization in Time 389

14.3 Separation of Variables 396

14.3.1 Separation of Variables for Difference Equations 400

14.4 Hyperbolic Equations 402

14.4.1 Characteristics 402

14.4.2 Systems of Hyperbolic Equations 403

14.4.3 Boundary Conditions 404

14.4.4 Finite Difference Methods 404

14.5 Fast Methods for Poisson's Equation 409

14.5.1 The Fast Fourier Transform 411

14.6 Multigrid Methods 414

14.7 Chapter 14 Exercises 418

APPENDIX A REVIEW OF LINEAR ALGEBRA 421

A.1 Vectors and Vector Spaces 421

A.2 Linear Independence and Dependence 422

A.3 Span of a Set of Vectors; Bases and Coordinates; Dimension of a Vector Space 423

A.4 The Dot Product; Orthogonal and Orthonormal Sets; the Gram-Schmidt Algorithm 423

A.5 Matrices and Linear Equations 425

A.6 Existence and Uniqueness of Solutions; the Inverse; Conditions for Invertibility 427

A.7 Linear Transformations; the Matrix of a Linear Transformation 431

A.8 Similarity Transformations; Eigenvalues and Eigenvectors 432

APPENDIX B TAYLOR'S THEOREM IN MULTIDIMENSIONS 436

References 439

Index 445