Matrix Analysis for Statistics

Chapter 1: A Review of Elementary Matrix Algebra

• • 1.1 Introduction
• • 1.2 Definitions and Notation
• • 1.3 Matrix Addition and Multiplication
• • 1.4 The Transpose
• • 1.5 The Trace
• • 1.6 The Determinant
• • 1.7 The Inverse
• • 1.8 Partitioned Matrices
• • 1.9 The Rank of a Matrix
• • 1.10 Orthogonal Matrices
• • 1.11 Quadratic Forms
• • 1.12 Complex Matrices
• • 1.13 Random Vectors and Some Related Statistical Concepts
• • Problems

Chapter 2: Vector Spaces

• • 2.1 Introduction
• • 2.2 Definitions
• • 2.3 Linear Independence and Dependence
• • 2.4 Matrix Rank and Linear Independence
• • 2.5 Bases and Dimension
• • 2.6 Orthonormal Bases and Projections
• • 2.7 Projection Matrices
• • 2.8 Linear Transformations and Systems of Linear Equations
• • 2.9 The Intersection and Sum of Vector Spaces
• • 2.10 Oblique Projections
• • 2.11 Convex Sets
• • Problems

Chapter 3: Eigenvalues and Eigenvectors

• • 3.1 Introduction
• • 3.2 Eigenvalues, Eigenvectors, and Eigenspaces
• • 3.3 Some Basic Properties of Eigenvalues and Eigenvectors
• • 3.4 Symmetric Matrices
• • 3.5 Continuity of Eigenvalues and Eigenprojections
• • 3.6 Extremal Properties of Eigenvalues
• • 3.7 Additional Results Concerning Eigenvalues Of Symmetric Matrices
• • 3.8 Nonnegative Definite Matrices
• • 3.9 Antieigenvalues and Antieigenvectors
• • Problems

Chapter 4: Matrix Factorizations and Matrix Norms

• • 4.1 Introduction
• • 4.2 The Singular Value Decomposition
• • 4.3 The Spectral Decomposition of a Symmetric Matrix
• • 4.4 The Diagonalization of a Square Matrix
• • 4.5 The Jordan Decomposition
• • 4.6 The Schur Decomposition
• • 4.7 The Simultaneous Diagonalization of Two Symmetric Matrices
• • 4.8 Matrix Norms
• • Problems

Chapter 5: Generalized Inverses

• • 5.1 Introduction
• • 5.2 The Moore–Penrose Generalized Inverse
• • 5.3 Some Basic Properties of the Moore–Penrose Inverse
• • 5.4 The Moore–Penrose Inverse of a Matrix Product
• • 5.5 The Moore–Penrose Inverse of Partitioned Matrices
• • 5.6 The Moore–Penrose Inverse of a Sum
• • 5.7 The Continuity of the Moore–Penrose Inverse
• • 5.8 Some Other Generalized Inverses
• • 5.9 Computing Generalized Inverses
• • Problems

Chapter 6: Systems of Linear Equations

• • 6.1 Introduction
• • 6.2 Consistency of a System of Equations
• • 6.3 Solutions to a Consistent System of Equations
• • 6.4 Homogeneous Systems of Equations
• • 6.5 Least Squares Solutions to a System of Linear Equations
• • 6.6 Least Squares Estimation For Less Than Full Rank Models
• • 6.7 Systems of Linear Equations and The Singular Value Decomposition
• • 6.8 Sparse Linear Systems of Equations
• • Problems

Chapter 7: Partitioned Matrices

• • 7.1 Introduction
• • 7.2 The Inverse
• • 7.3 The Determinant
• • 7.4 Rank
• • 7.5 Generalized Inverses
• • 7.6 Eigenvalues
• • Problems

Chapter 8: Special Matrices and Matrix Operations

• • 8.1 Introduction
• • 8.2 The Kronecker Product
• • 8.3 The Direct Sum
• • 8.4 The Vec Operator
• • 8.5 The Hadamard Product
• • 8.6 The Commutation Matrix
• • 8.7 Some Other Matrices Associated With the Vec Operator
• • 8.8 Nonnegative Matrices
• • 8.9 Circulant and Toeplitz Matrices
• • 8.10 Hadamard and Vandermonde Matrices
• • Problems

Chapter 9: Matrix Derivatives and Related Topics

• • 9.1 Introduction
• • 9.2 Multivariable Differential Calculus
• • 9.3 Vector and Matrix Functions
• • 9.4 Some Useful Matrix Derivatives
• • 9.5 Derivatives of Functions of Patterned Matrices
• • 9.6 The Perturbation Method
• • 9.7 Maxima and Minima
• • 9.8 Convex and Concave Functions
• • 9.9 The Method of Lagrange Multipliers
• • Problems

Chapter 10: Inequalities

• • 10.1 Introduction
• • 10.2 Majorization
• • 10.3 Cauchy-Schwarz Inequalities
• • 10.4 H¨older’s Inequality
• • 10.5 Minkowski’s Inequality
• • 10.6 The Arithmetic-Geometric Mean Inequality
• • Problems

Chapter 11: Some Special Topics Related to Quadratic Forms

• • 11.1 Introduction
• • 11.2 Some Results on Idempotent Matrices
• • 11.3 Cochran’s Theorem
• • 11.4 Distribution of Quadratic Forms in Normal Variates
• • 11.5 Independence of Quadratic Forms
• • 11.6 Expected Values of Quadratic Forms
• • 11.7 The Wishart Distribution
• • Problems