90. Pseudoinverse

The Moore–Penrose pseudoinverse ${𝐴}^{+}$ generalizes the matrix inverse to every matrix — square, rectangular, full-rank, or rank-deficient.

For invertible square $𝐴$, ${𝐴}^{+}={𝐴}^{-1}$. For other matrices, ${𝐴}^{+}$ provides the “best possible inverse” in a precise least-squares sense.

90.1. Definition via SVD

If $𝐴=𝑈\mathrm{\Sigma }{𝑉}^{𝑇}$ is the SVD of $𝐴$, then:

where ${\mathrm{\Sigma }}^{+}$ is the transpose of $\mathrm{\Sigma }$ with each non-zero singular value ${𝜎}_{𝑖}$ replaced by $\frac{1}{{𝜎}_{𝑖}}$ (zero singular values stay zero).

90.2. Four defining properties

${𝐴}^{+}$ is uniquely characterized by the Moore–Penrose conditions:

1. $𝐴{𝐴}^{+}𝐴=𝐴$
2. ${𝐴}^{+}𝐴{𝐴}^{+}={𝐴}^{+}$
3. ${\left(𝐴{𝐴}^{+}\right)}^{𝑇}=𝐴{𝐴}^{+}$ — symmetric
4. ${\left({𝐴}^{+}𝐴\right)}^{𝑇}={𝐴}^{+}𝐴$ — symmetric

These force a unique ${𝐴}^{+}$ for every matrix.

90.3. Special cases

Full column rank ($𝑚\ge 𝑛$, $\text{rank}\left(𝐴\right)=𝑛$):

Full row rank ($𝑚\le 𝑛$, $\text{rank}\left(𝐴\right)=𝑚$):

Square invertible: ${𝐴}^{+}={𝐴}^{-1}$.

90.4. Least-squares solution

For the overdetermined system $𝐴\overrightarrow{𝑥}=\overrightarrow{𝑏}$ (more equations than unknowns), the minimum-norm least-squares solution is:

Properties:

• If a solution to $𝐴\overrightarrow{𝑥}=\overrightarrow{𝑏}$ exists, $\widehat{\overrightarrow{𝑥}}$ is one (the one with smallest norm)
• If no exact solution exists, $\widehat{\overrightarrow{𝑥}}$ minimizes $\Vert 𝐴\overrightarrow{𝑥}-\overrightarrow{𝑏}\Vert$

This is the foundation of linear regression and least-squares fitting.

90.5. Why it generalizes the inverse

• Replaces “exact solution” with “best least-squares approximation”
• Handles rank-deficient systems (where standard inverse is undefined)
• Handles rectangular systems (where standard inverse is undefined)
• Reduces to ordinary inverse exactly when ordinary inverse exists

90.6. See also

• SVD — the construction
• Matrix Inverse — special case
• QR — used to compute the pseudoinverse in the full-rank case
• Linear Regression — pseudoinverse gives OLS coefficients $\widehat{𝛽}={\left({𝑋}^{𝑇}𝑋\right)}^{-1}{𝑋}^{𝑇}𝑦={𝑋}^{+}𝑦$