Background
Overview
An Urns Model for Remote Sensing
Hidden Markov Models
Measuring the Fourier Transform
Transmission Tomography
Emission Tomography
A Unifying Framework
Sequential Optimization
Overview
Examples of SUM
Auxiliary-Function Methods
The SUMMA Class of AF Methods
Barrier-Function and Penalty-Function Methods
Barrier Functions
Examples of Barrier Functions
Penalty Functions
Examples of Penalty Functions
Basic Facts
Proximal Minimization
The Basic Problem
Proximal Minimization Algorithms
Some Obstacles
All PMA Are SUMMA
Convergence of the PMA
The Non-Differentiable Case
The IPA
Projected Gradient Descent
Relaxed Gradient Descent
Regularized Gradient Descent
The Projected Landweber Algorithm
The Simultaneous MART
A Convergence Theorem
Another Job for the PMA
The Goldstein-Osher Algorithm
A Question
The Forward-Backward Splitting Algorithm
Moreau’s Proximity Operators
The FBS Algorithm
Convergence of the FBS Algorithm
Some Examples
Minimizing f2 over a Linear Manifold
Feasible-Point Algorithms
Operators
Overview
Operators
Contraction Operators
Convex Sets in RJ
Orthogonal Projection Operators
Firmly Nonexpansive Gradients
Exercises
Averaged and Paracontractive Operators
Averaged Operators
Gradient Operators
Two Useful Identities
The Krasnosel’skii-Mann-Opial Theorem
Affine Linear Operators
Paracontractive Operators
Exercises
Convex Feasibility and Related Problems
Convex Constraint Sets
Using Orthogonal Projections
The ART
Regularization
Avoiding the Limit Cycle
Exercises
Eigenvalue Bounds
Introduction and Notation
Overview
Cimmino’s Algorithm
The Landweber Algorithms
Some Upper Bounds for L
Simultaneous Iterative Algorithms
Block-Iterative Algorithms
Exercises
Jacobi and Gauss-Seidel Methods
The Jacobi and Gauss-Seidel Methods: An Example
Splitting Methods
Some Examples of Splitting Methods
Jacobi’s Algorithm and JOR
The Gauss-Seidel Algorithm and SOR
Summary
The SMART and EMML Algorithms
The SMART Iteration
The EMML Iteration
The EMML and the SMART as AM
The SMART as SUMMA
The SMART as PMA
Using KL Projections
The MART and EMART Algorithms
Extensions of MART and EMART
Convergence of the SMART and EMML
Regularization
Modifying the KL Distance
The ABMART Algorithm
The ABEMML Algorithm
Alternating Minimization
Alternating Minimization
Exercises
The EM Algorithm
Overview
A Non-Stochastic Formulation of EM
The Stochastic EM Algorithm
The Discrete Case
Missing Data
The Continuous Case
EM and the KL Distance
Finite Mixture Problems
Geometric Programming and the MART
Overview
An Example of a GP Problem
The Generalized AGM Inequality
Posynomials and the GP Problem
The Dual GP Problem
Solving the GP Problem
Solving the DGP Problem
Constrained Geometric Programming
Exercises
Variational Inequality Problems and Algorithms
Monotone Functions
The Split-Feasibility Problem
The Variational Inequality Problem
Korpelevich’s Method for the VIP
On Some Algorithms of Noor
Split Variational Inequality Problems
Saddle Points
Exercises
Set-Valued Functions in Optimization
Overview
Notation and Definitions
Basic Facts
Monotone Set-Valued Functions
Resolvents
Split Monotone Variational Inclusion
Solving the SMVIP
Special Cases of the SMVIP
The Split Common Null-Point Problem
Exercises
Fenchel Duality
The Legendre-Fenchel Transformation
Fenchel’s Duality Theorem
An Application to Game Theory
Exercises
Compressed Sensing
Compressed Sensing
Sparse Solutions
Minimum One-Norm Solutions
Why Sparseness?
Compressed Sampling
Appendix: Bregman-Legendre Functions
Essential Smoothness and Essential Strict Convexity
Bregman Projections onto Closed Convex Sets
Bregman-Legendre Functions
Bibliography
Index