Inner Product

10. Inner Product

An inner product is a function $⟨ \cdot, \cdot ⟩ : 𝑉 \times 𝑉 \to ℝ$ that generalizes the dot product to arbitrary vector spaces.

10.1. Axioms

For all $\vec{𝑢}, \vec{𝑣}, \vec{𝑤} \in 𝑉$ and scalar $𝑐 \in ℝ$ :

Symmetry: $⟨ \vec{𝑢}, \vec{𝑣} ⟩ = ⟨ \vec{𝑣}, \vec{𝑢} ⟩$
Linearity in the first argument:

⟨ 𝑐 \vec{𝑢} + \vec{𝑤}, \vec{𝑣} ⟩ = 𝑐 ⟨ \vec{𝑢}, \vec{𝑣} ⟩ + ⟨ \vec{𝑤}, \vec{𝑣} ⟩

(by symmetry, also linear in the second argument)

Positive-definiteness: $⟨ \vec{𝑣}, \vec{𝑣} ⟩ \geq 0$ , with equality only when $\vec{𝑣} = 𝟎$

A vector space with an inner product is an inner product space.

10.2. Standard examples

Dot product on $ℝ^{𝑛}$ : $⟨ \vec{𝑢}, \vec{𝑣} ⟩ = \sum_{𝑖 = 1}^{𝑛} 𝑢_{𝑖} 𝑣_{𝑖} = {\vec{𝑢}}^{𝑇} \vec{𝑣}$

Weighted inner product: ${⟨ \vec{𝑢}, \vec{𝑣} ⟩}_{𝑊} = {\vec{𝑢}}^{𝑇} 𝑊 \vec{𝑣}$ where $𝑊$ is symmetric positive-definite

Function space inner product (continuous functions on $[𝑎, 𝑏]$ ):

⟨ 𝑓, 𝑔 ⟩ = \int_{𝑎}^{𝑏} 𝑓 (𝑥) 𝑔 (𝑥) 𝑑 𝑥

Frobenius inner product on matrices:

{⟨ 𝐴, 𝐵 ⟩}_{𝐹} = tr (𝐴^{𝑇} 𝐵) = \sum_{𝑖, 𝑗} 𝐴_{𝑖 𝑗} 𝐵_{𝑖 𝑗}

10.3. Induced norm

Every inner product induces a norm:

‖ \vec{𝑣} ‖ = \sqrt{⟨ \vec{𝑣}, \vec{𝑣} ⟩}

This norm satisfies the parallelogram law $‖ \vec{𝑢} + \vec{𝑣} ‖^{2} + ‖ \vec{𝑢} - \vec{𝑣} ‖^{2} = 2 ‖ \vec{𝑢} ‖^{2} + 2 ‖ \vec{𝑣} ‖^{2}$ — a necessary and sufficient condition for a norm to come from an inner product.

10.4. Cauchy–Schwarz (general form)

The Cauchy–Schwarz inequality holds in every inner product space:

| ⟨ \vec{𝑢}, \vec{𝑣} ⟩ | \leq ‖ \vec{𝑢} ‖ ‖ \vec{𝑣} ‖

10.5. Orthogonality

Two vectors are orthogonal if their inner product is zero:

\vec{𝑢} ⟂ \vec{𝑣} \Leftrightarrow ⟨ \vec{𝑢}, \vec{𝑣} ⟩ = 0

This generalizes perpendicularity in $ℝ^{𝑛}$ . See Orthogonality.

10.6. Angle between vectors

cos 𝜃 = \frac{⟨ \vec{𝑢}, \vec{𝑣} ⟩}{‖ \vec{𝑢} ‖ ‖ \vec{𝑣} ‖}

In abstract spaces this defines the angle (between non-zero vectors).

10.7. Why inner products matter

Generalize geometry (length, angle, projection) to any vector space
Function spaces: e.g., Hilbert spaces in functional analysis use $⟨ 𝑓, 𝑔 ⟩ = \int 𝑓 𝑔$
Quantum mechanics: state space is a complex inner product space (Hilbert space)
Statistics: covariance is an inner product on centered random variables
Machine learning: kernel methods replace explicit inner products with implicit ones via kernels

10.8. See also

Dot Product — special case in $ℝ^{𝑛}$
Norm — derived from the inner product
Orthogonality
Gram–Schmidt Process — orthogonalization via inner products
Cauchy–Schwarz Inequality