89. SVD

Every real matrix — square or rectangular, full-rank or rank-deficient — has a Singular Value Decomposition:

where, for an matrix :

• is orthogonal — columns are the left singular vectors
• is diagonal with non-negative entries — the singular values
• is orthogonal — columns are the right singular vectors

89.1. Singular values

The singular values are the square roots of the eigenvalues of (which is symmetric positive semi-definite):

Number of non-zero singular values = rank of .

89.2. Geometric interpretation

Every linear transformation decomposes into:

1. Rotate / reflect in the input space (multiply by )
2. Stretch along the new axes by factors (multiply by )
3. Rotate / reflect in the output space (multiply by )

So every matrix is a rotation–stretch–rotation. The singular values measure how much stretches each direction.

89.3. Compact / reduced SVD

For rank- matrix , drop the zero singular values:

where is , is diagonal, is . Same product, less storage.

89.4. Best low-rank approximation

The truncated SVD (keeping the top singular values) is the best rank- approximation of in both Frobenius and spectral norms:

This is the engine behind:

• Image / signal compression
• Recommender systems / matrix factorization
• Noise reduction — drop tiny singular values

89.5. Connection to symmetric matrices

If is symmetric positive semi-definite, the SVD coincides with the eigendecomposition (Spectral Theorem): and = eigenvalues.

For general , and are both symmetric:

• Right singular vectors of = eigenvectors of (with eigenvalues )
• Left singular vectors of = eigenvectors of (with same eigenvalues )

89.6. Connection to pseudoinverse

When is not square or not invertible, the Moore–Penrose pseudoinverse uses the SVD:

where inverts the non-zero singular values and transposes the shape.

89.7. See also

• Spectral Theorem — symmetric special case
• Pseudoinverse
• QR / Cholesky / LU — other matrix factorizations
• Orthogonal Matrix