Joint, Marginal & Conditional Probability

146. Joint, Marginal & Conditional Probability

Setup

Imagine we have 100 emails. We track two things about each one: is it spam or not, and does it contain the word “free” or not. Here’s the count of every combination:

	$𝐹$	$𝐹^{𝑐}$	margin
$𝑆$	0.20	0.10	0.30
$𝑆^{𝑐}$	0.05	0.65	0.70
margin	0.25	0.75	1.00

146.1. Joint Probability

𝑃 (𝑆 \cap 𝐹) = 0.20

146.2. Marginal Probability

𝑃 (𝐴) = \sum_{𝑖 = 1}^{𝑛} 𝑃 (𝐴 \cap 𝐵_{𝑖})

\begin{aligned} 𝑃 (𝑆) & = 𝑃 (𝑆 \cap 𝐹) + 𝑃 (𝑆 \cap 𝐹^{𝑐}) \\ = 0.20 + 0.10 \\ = 0.30 \end{aligned}

146.3. Conditional Probability

𝑃 (𝐴 | 𝐵) = \frac{𝑃 (𝐴 \cap 𝐵)}{𝑃 (𝐵)}

𝑃 (𝑆 | 𝐹) = \frac{𝑃 (𝑆 \cap 𝐹)}{𝑃 (𝐹)} = \frac{0.20}{0.25} = 0.80

146.4. Convolution

You have two independent random variables, $𝑋$ and $𝑌$ . You add them: $𝑍 = 𝑋 + 𝑌$ . What’s the distribution of $𝑍$ ?

146.4.1. Discrete

𝑃 (𝑍 = 𝑧) = \sum_{𝑥} 𝑃 (𝑋 = 𝑥) 𝑃 (𝑌 = 𝑧 - 𝑥)

146.4.2. Continuous

𝑓_{𝑍} (𝑧) = \int_{- \infty}^{+ \infty} 𝑓_{𝑋} (𝑥) 𝑓_{𝑌} (𝑧 - 𝑥)