94. Spectral Theorem
The Spectral Theorem says that every real symmetric matrix is orthogonally diagonalizable:
where:
- is a diagonal matrix whose entries are the (always real) eigenvalues of
- is an orthogonal matrix whose columns are an orthonormal basis of eigenvectors of
Equivalently, , and the columns of are orthonormal eigenvectors.
94.1. What it guarantees
For a real symmetric matrix :
- All eigenvalues are real (no complex eigenvalues, even if ‘s entries don’t suggest it)
- Eigenvectors for distinct eigenvalues are orthogonal (automatic)
- Eigenvectors for repeated eigenvalues can be chosen orthonormal (within that eigenspace, via Gram–Schmidt)
- The full set of orthonormal eigenvectors forms a basis of
The trio (1)+(2)+(3) is what makes the theorem work. No real symmetric matrix is missing eigenvectors.
94.2. Spectral expansion
Write and . Then:
The matrix decomposes into a sum of rank-1 spectral projectors , each scaled by an eigenvalue. This is the spectral decomposition.
Example
Eigenvalues: .
Orthonormal eigenvectors: , .
Spectral expansion: .
94.3. Generalizations
- Complex Hermitian matrices (, where is the conjugate transpose): same theorem, with replaced by a unitary matrix ().
- Normal matrices (): same theorem in — unitarily diagonalizable.
94.4. Why it matters
- Quadratic forms: simplifies to in the eigenbasis . Definiteness is read off the signs of eigenvalues.
- PCA / dimensionality reduction: the covariance matrix is symmetric → orthogonal eigenbasis = principal components.
- Optimization: the Hessian is symmetric → its spectrum classifies critical points.
- Quantum mechanics: observables are Hermitian operators; their eigenvalues are the possible measured values.
- SVD extends the idea to non-square / non-symmetric matrices.