Covariance & Correlation

210. Covariance & Correlation

210.1. Covariance

\begin{matrix} Cov (𝑋, 𝑌) = 𝔼 [(𝑋 - 𝔼 [𝑋]) (𝑌 - 𝔼 [𝑌])] = 𝔼 [𝑋 𝑌] - 𝔼 [𝑋] 𝔼 [𝑌] \\ \frac{\sum_{𝑖 = 1}^{𝑛} (𝑥_{𝑖} - \bar{𝑥}) (𝑦_{𝑖} - \bar{𝑦})}{𝑛} \end{matrix}

Example

\begin{matrix} 𝑋 = (1, 2, 3) \\ 𝑌 = (2, 4, 6) \end{matrix}

\begin{matrix} 𝔼 [𝑋] = \frac{1 + 2 + 3}{3} = 2 \\ 𝔼 [𝑌] = \frac{2 + 4 + 6}{3} = 4 \end{matrix}

$𝑥_{𝑖}$	$𝑦_{𝑖}$	$𝑥_{𝑖} - 𝔼 [𝑋]$	$𝑦_{𝑖} - 𝔼 [𝑌]$	product
1	2	−1	−2	2
2	4	0	0	0
3	6	1	2	2

Cov (𝑋, 𝑌) = \frac{2 + 0 + 2}{3} = \frac{4}{3} \approx 1.33

Cross-check with the formula:

𝔼 [𝑋 𝑌] - 𝔼 [𝑋] 𝔼 [𝑌]

\begin{matrix} 𝔼 [𝑋 𝑌] = \frac{(1) (2) + (2) (4) + (3) (6)}{3} = \frac{2 + 8 + 18}{3} = \frac{28}{3} \\ 𝔼 [𝑋] 𝔼 [𝑌] = (2) (4) = 8 \\ 𝔼 [𝑋 𝑌] - 𝔼 [𝑋] 𝔼 [𝑌] = \frac{28}{3} - 8 = \frac{4}{3} \end{matrix}

Covariance is unbounded; correlation is the normalized version:

𝜌_{𝑋 𝑌} = \frac{Cov (𝑋, 𝑌)}{𝜎_{𝑋} 𝜎_{𝑌}}

210.2. Covariance and Correlation