Projection

70. Projection

The projection of a vector $\vec{𝑥}$ , onto a line $𝐿$ , denoted as ${Proj}_{𝐿} (\vec{𝑥})$ , is a vector that lies on the line $𝐿$ , such that the difference between $\vec{𝑥}$ and its projection, ${Proj}_{𝐿} (\vec{𝑥}) - \vec{𝑥}$ , is orthogonal to $𝐿$

${Proj}_{𝐿} (\vec{𝑥})$ can be seen as the “shadow”cast by $\vec{𝑥}$ onto $𝐿$ when light shines perpendicularly to $𝐿$ .

{Proj}_{𝐿} (\vec{𝑥}) = 𝑐 \vec{𝑣} = (\frac{\vec{𝑥} \cdot \vec{𝑣}}{\vec{𝑣} \cdot \vec{𝑣}}) \vec{𝑣}

Equivalently

{Proj}_{𝐿} (\vec{𝑥}) = 𝑐 \vec{𝑣} = (\frac{\vec{𝑥} \cdot \vec{𝑣}}{‖ \vec{𝑣} ‖^{2}}) \vec{𝑣}

Where:

$\vec{𝑣}$ is a direction vector for the line $𝐿$
$𝑐 = \frac{\vec{𝑥} \cdot \vec{𝑣}}{\vec{𝑣} \cdot \vec{𝑣}}$ is a scalar

Example

\vec{𝑥} = [\begin{matrix} 2 \\ 3 \end{matrix}], \vec{𝑣} = [\begin{matrix} 2 \\ 1 \end{matrix}]

Define the line $𝐿$ as all vectors of the form $𝑐 \vec{𝑣}$ , where $𝑐$ is a scalar:

\begin{aligned} 𝐿 & = {𝑐 \vec{𝑣} | 𝑐 \in ℝ} \\ = {𝑐 [\begin{matrix} 2 \\ 1 \end{matrix}] | 𝑐 \in ℝ} \end{aligned}

Compute the projection using

\begin{aligned} {Proj}_{𝐿} (\vec{𝑥}) & = \frac{\vec{𝑥} \cdot \vec{𝑣}}{\vec{𝑣} \cdot \vec{𝑣}} \vec{𝑣} \\ = \frac{[\begin{matrix} 2 \\ 3 \end{matrix}] \cdot [\begin{matrix} 2 \\ 1 \end{matrix}]}{[\begin{matrix} 2 \\ 1 \end{matrix}] \cdot [\begin{matrix} 2 \\ 1 \end{matrix}]} [\begin{matrix} 2 \\ 1 \end{matrix}] \\ = \frac{7}{5} [\begin{matrix} 2 \\ 1 \end{matrix}] \\ = [\begin{matrix} 2.8 \\ 1.4 \end{matrix}] \end{aligned}

Projection as a Transformation

As a matrix vector product

𝐿 = {𝑐 \vec{𝑣} | 𝑐 \in ℝ}

{Proj}_{𝐿} : ℝ^{𝑛} \to ℝ^{𝑛}

\vec{𝑣} \cdot \vec{𝑣} = ‖ \vec{𝑣} ‖^{2}

If $\vec{𝑣}$ is a unit vector:

‖ \vec{𝑣} ‖ = 1

Then

\begin{aligned} {Proj}_{𝐿} (\vec{𝑥}) & = (\frac{\vec{𝑥} \cdot \vec{𝑣}}{\vec{𝑣} \cdot \vec{𝑣}}) \vec{𝑣} \\ = (\frac{\vec{𝑥} \cdot \vec{𝑣}}{‖ \vec{𝑣} ‖^{2}}) \vec{𝑣} \end{aligned}

If we redefine our line $𝐿$ as all the scalar multiples of out unit vector $\hat{𝑢}$ :

𝐿 = {𝑐 \hat{𝑢} | 𝑐 \in ℝ}

Simplifies to:

(\vec{𝑥} \cdot \hat{𝑢}) \hat{𝑢}

Projection as a Linear Transformation

Let $\hat{𝑢} \in ℝ^{𝑛}$ be a unit vector:

\hat{𝑢} = [\begin{matrix} 𝑢_{1} \\ 𝑢_{2} \\ ⋮ \\ 𝑢_{𝑛} \end{matrix}], where ‖ \hat{𝑢} ‖ = 1

The projection matrix $𝐴$ is:

𝐴 = \hat{𝑢} {\hat{𝑢}}^{𝑇} = [\begin{matrix} 𝑢_{1} \\ 𝑢_{2} \\ ⋮ \\ 𝑢_{𝑛} \end{matrix}] [𝑢_{1} 𝑢_{2} \dots 𝑢_{𝑛}]

Expands to:

𝐴 = [\begin{matrix} 𝑢_{1} 𝑢_{1} & 𝑢_{1} 𝑢_{2} & \dots & 𝑢_{1} 𝑢_{𝑛} \\ 𝑢_{2} 𝑢_{1} & 𝑢_{2} 𝑢_{2} & \dots & 𝑢_{2} 𝑢_{𝑛} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 𝑢_{𝑛} 𝑢_{1} & 𝑢_{𝑛} 𝑢_{2} & \dots & 𝑢_{𝑛} 𝑢_{𝑛} \end{matrix}]

For any $\vec{𝑥} \in ℝ^{𝑛}$ , the projection of $\vec{𝑥}$ onto the line spanned by $\hat{𝑢}$ is:

{Proj}_{𝐿} (\vec{𝑥}) = 𝐴 \vec{𝑥} = (\hat{𝑢} \cdot \vec{𝑥}) \hat{𝑢}

Example

Consider a vector $\vec{𝑣}$ in $ℝ^{2}$ :

\vec{𝑣} = [\begin{matrix} 2 \\ 1 \end{matrix}]

1. Construct the Unit Vector

‖ \vec{𝑣} = \sqrt{2^{2} + 1^{2}} = \sqrt{5}

\hat{𝑢} = \frac{\vec{𝑣}}{‖ \vec{𝑣} ‖} = \frac{1}{\sqrt{5}} [\begin{matrix} 2 \\ 1 \end{matrix}] = [\begin{matrix} \frac{2}{\sqrt{5}} \\ \frac{1}{\sqrt{5}} \end{matrix}]

This unit vector $\hat{𝑢}$ defines the line $𝐿$ , which consists of all the scalar multiples og $\vec{𝑣}$ :

𝐿 = {𝑐 \vec{𝑣} | 𝑐 \in ℝ}

2. Derive the Projection Matrix

The projection of any vector $\vec{𝑥}$ onto the line $𝐿$ is given by:

{Proj}_{𝐿} (\vec{𝑥}) = 𝐴 \vec{𝑥}

Where $𝐴$ is the projection matrix. To construct $𝐴$ we use the formula:

\begin{aligned} 𝐴 & = \hat{𝑢} {\hat{𝑢}}^{𝑇} \\ = [\begin{matrix} 𝑢_{1} \\ 𝑢_{2} \end{matrix}] [𝑢_{1} 𝑢_{2}] \\ = [\begin{matrix} 𝑢_{1} \cdot 𝑢_{1} & 𝑢_{1} \cdot 𝑢_{2} \\ 𝑢_{2} \cdot 𝑢_{1} & 𝑢_{2} \cdot 𝑢_{2} \end{matrix}] \\ = [\begin{matrix} \frac{2}{\sqrt{5}} \\ \frac{1}{\sqrt{5}} \end{matrix}] [\frac{2}{\sqrt{5}} \frac{1}{\sqrt{5}}] \\ = [\begin{matrix} {(\frac{2}{\sqrt{5}})}^{2} & \frac{1}{\sqrt{5}} \frac{2}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} \frac{1}{\sqrt{5}} & {(\frac{1}{\sqrt{5}})}^{2} \end{matrix}] \\ = [\begin{matrix} \frac{4}{5} & \frac{2}{5} \\ \frac{2}{5} & \frac{1}{5} \end{matrix}] \end{aligned}

3. Applying the Projection

To project any vector $\vec{𝑥}$ onto $𝐿$ , we multiply $\vec{𝑥}$ by the matrix $𝐴$ :

\begin{aligned} {Proj}_{𝐿} (\vec{𝑥}) & = 𝐴 \vec{𝑥} \\ = [\begin{matrix} \frac{4}{5} & \frac{2}{5} \\ \frac{2}{5} & \frac{1}{5} \end{matrix}] \vec{𝑥} \end{aligned}

Additivity of Projections (Linearity with respect to addition)

\begin{aligned} {Proj}_{𝐿} (\vec{𝑎} + \vec{𝑏}) & = ((\vec{𝑎} + \vec{𝑏}) \cdot \hat{𝑢}) \\ = (\vec{𝑎} \cdot \hat{𝑢} + \vec{𝑏} \cdot \hat{𝑢}) \hat{𝑢} \\ = (\vec{𝑎} \cdot \hat{𝑢}) \hat{𝑢} + (\vec{𝑏} \cdot \hat{𝑢}) \hat{𝑢} \\ = {Proj}_{𝐿} (\vec{𝑎}) + {Proj}_{𝐿} (\vec{𝑏}) \end{aligned}

Homogeneity of Projections (Linearity with respect to scalar multiplication)

\begin{aligned} {Proj}_{𝐿} (𝑐 \vec{𝑎}) & = (𝑐 \vec{𝑎} \cdot \hat{𝑢}) \hat{𝑢} \\ = 𝑐 (\vec{𝑎} \cdot \hat{𝑢}) \hat{𝑢} \\ = 𝑐 {Proj}_{𝐿} (\vec{𝑎}) \end{aligned}

General Properties of $𝐴$

Idempotence

𝐴^{2} = 𝐴

Symmetry

𝐴^{𝑇} = 𝐴

Rank

rank (𝐴) = 1

Because $\hat{𝑢} {\hat{𝑢}}^{𝑇}$ projects onto a one-dimensional subspace spanned by $\hat{𝑢}$