Table of Contents
Preface xi
Global References xiii
Other Books xv
Useful URLs xix
Common Notation xxi
1 Matrix Multiplication 1
1.1 Basic Algorithms and Notation 2
1.2 Structure and Efficiency 14
1.3 Block Matrices and Algorithms 22
1.4 Fast Matrix-Vector Products 33
1.5 Vectorization and Locality 43
1.6 Parallel Matrix Multiplication 49
2 Matrix Analysis 63
2.1 Basic Ideas from Linear Algebra 64
2.2 Vector Norms 68
2.3 Matrix Norms 71
2.4 The Singular Value Decomposition 76
2.5 Subspace Metrics 81
2.6 The Sensitivity of Square Systems 87
2.7 Finite Precision Matrix Computations 93
3 General Linear Systems 105
3.1 Triangular Systems 106
3.2 The LU Factorization 111
3.3 Round off Error in Gaussian Elimination 122
3.4 Pivoting 125
3.5 Improving and Estimating Accuracy 137
3.6 Parallel LU 144
4 Special Linear Systems 153
4.1 Diagonal Dominance and Symmetry 154
4.2 Positive Definite Systems 159
4.3 Banded Systems 176
4.4 Symmetric Indefinite Systems 186
4.5 Block Tridiagonal Systems 196
4.6 Vandermonde Systems 203
4.7 Classical Methods for Toeplitz Systems 208
4.8 Circulant and Discrete Poisson Systems 219
5 Orthogonalization and Least Squares 233
5.1 Householder and Givens Transformations 234
5.2 The QR Factorization 246
5.3 The Full-Rank Least Squares Problem 260
5.4 Other Orthogonal Factorizations 274
5.5 The Rank-Deficient Least Squares Problem 288
5.6 Square and Underdetermined Systems 298
6 Modified Least Squares Problems and Methods 303
6.1 Weighting and Regularization 304
6.2 Constrained Least Squares 313
6.3 Total Least Squares 320
6.4 Subspace Computations with the SVD 327
6.5 Updating Matrix Factorizations 334
7 Unsymmetric Eigenvalue Problems 347
7.1 Properties and Decompositions 348
7.2 Perturbation Theory 357
7.3 Power Iterations 365
7.4 The Hessenberg and Real Schur Forms 376
7.5 The Practical QR Algorithm 385
7.6 Invariant Subspace Computations 394
7.7 The Generalized Eigenvalue Problem 405
7.8 Hamiltonian and Product Eigenvalue Problems 420
7.9 Pseudospectra 426
8 Symmetric Eigenvalue Problems 439
8.1 Properties and Decompositions 440
8.2 Power Iterations 450
8.3 The Symmetric QR Algorithm 458
8.4 More Methods for Tridiagonal Problems 467
8.5 Jacobi Methods 476
8.6 Computing the SVD 486
8.7 Generalized Eigenvalue Problems with Symmetry 497
9 Functions of Matrices 513
9.1 Eigenvalue Methods 514
9.2 Approximation Methods 522
9.3 The Matrix Exponential 530
9.4 The Sign, Square Root, and Log of a Matrix 536
10 Large Sparse Eigenvalue Problems 545
10.1 The Symmetric Lanczos Process 546
10.2 Lanczos, Quadrature, and Approximation 556
10.3 Practical Lanczos Procedures 562
10.4 Large Sparse SVD Frameworks 571
10.5 Krylov Methods for Unsymmetric Problems 579
10.6 Jacobi-Davidson and Related Methods 589
11 Large Sparse Linear System Problems 597
11.1 Direct Methods 598
11.2 The Classical Iterations 611
11.3 The Conjugate Gradient Method 625
11.4 Other Krylov Methods 639
11.5 Preconditioning 650
11.6 The Multigrid Framework 670
12 Special Topics 681
12.1 Linear Systems with Displacement Structure 681
12.2 Structured-Rank Problems 691
12.3 Kronecker Product Computations 707
12.4 Tensor Unfoldings and Contractions 719
12.5 Tensor Decompositions and Iterations 731
Index 747
