Norm

7. Norm

A scalar that measures the length (or magnitude) of a vector.

Notation: $‖ \vec{𝑣} ‖$ .

The default norm — Pythagoras applied componentwise.

For $\vec{𝑣} = [\begin{matrix} 𝑣_{1} \\ 𝑣_{2} \\ ⋮ \\ 𝑣_{𝑛} \end{matrix}] \in ℝ^{𝑛}$ :

‖ \vec{𝑣} ‖ = \sqrt{𝑣_{1}^{2} + 𝑣_{2}^{2} + \dots + 𝑣_{𝑛}^{2}}

Example

\vec{𝑣} = [\begin{matrix} 3 \\ 4 \end{matrix}] ‖ \vec{𝑣} ‖ = \sqrt{3^{2} + 4^{2}} = \sqrt{25} = 5

Geometrically, this is the distance from the origin to the point $(3, 4)$ .

The Euclidean norm is the dot product of a vector with itself, square-rooted:

‖ \vec{𝑣} ‖ = \sqrt{\vec{𝑣} \cdot \vec{𝑣}}

This connection makes norms central to projections, angles between vectors, and least-squares regression.

For any vectors $\vec{𝑢}, \vec{𝑣} \in ℝ^{𝑛}$ and scalar $𝑐 \in ℝ$ :

Non-negativity: $‖ \vec{𝑣} ‖ \geq 0$
Definiteness: $‖ \vec{𝑣} ‖ = 0 \Leftrightarrow \vec{𝑣} = 𝟎$
Absolute homogeneity: $‖ 𝑐 \vec{𝑣} ‖ = | 𝑐 | ‖ \vec{𝑣} ‖$
Triangle inequality: $‖ \vec{𝑢} + \vec{𝑣} ‖ \leq ‖ \vec{𝑢} ‖ + ‖ \vec{𝑣} ‖$ (see Triangle Inequality)

Any function $ℝ^{𝑛} \to ℝ$ satisfying these four axioms is a norm.

$‖ \vec{𝑣} ‖$ = distance from the origin to the point $\vec{𝑣}$
$‖ \vec{𝑢} - \vec{𝑣} ‖$ = distance between the two points $\vec{𝑢}$ and $\vec{𝑣}$
The set ${\vec{𝑥} : ‖ \vec{𝑥} ‖ = 1}$ is the unit sphere (a circle in $ℝ^{2}$ , sphere in $ℝ^{3}$ )

For $𝑝 \geq 1$ :

‖ \vec{𝑣} ‖_{𝑝} = {(\sum_{𝑖 = 1}^{𝑛} | 𝑣_{𝑖} |^{𝑝})}^{\frac{1}{𝑝}}

Special cases:

Example

$\vec{𝑣} = [\begin{matrix} 3 \\ - 4 \end{matrix}]$ :

\begin{matrix} ‖ \vec{𝑣} ‖_{1} = | 3 | + | - 4 | = 7 \\ ‖ \vec{𝑣} ‖_{2} = \sqrt{9 + 16} = 5 \\ ‖ \vec{𝑣} ‖_{\infty} = max (3, 4) = 4 \end{matrix}

Dividing a vector by its norm gives a unit vector in the same direction:

\hat{𝑣} = \frac{\vec{𝑣}}{‖} \vec{𝑣} ‖, ‖ \hat{𝑣} ‖ = 1

Dot Product — $‖ \vec{𝑣} ‖^{2} = \vec{𝑣} \cdot \vec{𝑣}$
Cauchy–Schwarz Inequality — $| \vec{𝑢} \cdot \vec{𝑣} | \leq ‖ \vec{𝑢} ‖ ‖ \vec{𝑣} ‖$
Triangle Inequality — $‖ \vec{𝑢} + \vec{𝑣} ‖ \leq ‖ \vec{𝑢} ‖ + ‖ \vec{𝑣} ‖$
Unit Vector — vectors with $‖ \vec{𝑣} ‖ = 1$
Angles Between Vectors — $cos (𝜃) = \frac{\vec{𝑢} \cdot \vec{𝑣}}{‖ \vec{𝑢} ‖ ‖ \vec{𝑣} ‖}$