Moment Generating Function

151. Moment Generating Function

For a random variable $𝑋$ , its MGF is:

\begin{aligned} 𝐺 (𝑡) & = \sum_{𝑖 = 1}^{\infty} \frac{𝐺^{(𝑘)} (0)}{𝑘!} \\ = \sum_{𝑖 = 1}^{\infty} \frac{𝐸 [𝑋^{𝑘}]}{𝑘!} 𝑡^{𝑘} \\ = 𝐸 [\sum_{𝑖 = 1}^{\infty} \frac{𝑋^{𝑘} 𝑡^{𝑘}}{𝑘!}] \\ = 𝐸 [\sum_{𝑖 = 1}^{\infty} \frac{{(𝑡 𝑋)}^{𝑘}}{𝑘!}] \\ = 𝐸 [𝑒^{𝑡 𝑋}] \end{aligned}

Therefore:

𝑀_{𝑋} (𝑡) = 𝐸 [𝑒^{𝑡 𝑋}]

In general $𝑀_{𝑋}^{𝑘} (0) = 𝐸 [𝑋^{𝑘}]$ . So you turn the crank – take $𝑘$ derivatives, plug in $𝑡 = 0$ and the $𝑘$ -th moment falls out. The whole sequence of moments (mean via the first, variance via the second, skewness via the third, …) is encoded in this one function, recoverable by differentiation.

Moment	Name	$𝑓$	Definition
1st	Mean	$𝐺^{1} (0) = 𝐸 [𝑋^{𝑘}]$	where the distribution sits
2nd	Variance	$𝐺^{2} (0) =$	How spread out it is
3rd	Skewness	$𝐺^{3} (0) =$	How lopsided it is (longer left or right tail)
4th	Kurtosis	$𝐺^{4} (0) =$	How heavy the tails are, how peaked
5th …	No names	$𝐺^{(𝑘)} (0) =$	No intuitive picture

Example