Uniform from $a$ to $b$

Discrete:

$$\begin{aligned} p_X(x) &= \frac{1}{b-a+1} \quad \mathbb{E}[X] = \frac{a+b}{2} \quad var(X) = \frac{1}{12}(b-a)\cdot(b-a\color{blue}{+2}) \\ \mathbb{P}(a \le x \le b) &= \sum_{a \le x \le b}p_X(x) \\ \end{aligned}$$

Continuous:

$$\begin{aligned} f_X(x) &= \frac{1}{b-a} \quad \mathbb{E}[X] = \frac{a+b}{2} \quad var(X) = \frac{(b-a)^2}{12} \\ \mathbb{P}(a \le x \le b) &= \int_a^b f_X(x)\,dx \\ \end{aligned}$$

Bernoulli with parameter $p \in [0, 1]$

$$\begin{aligned} p_X(0) = 1-p \quad p_X(1) = p \quad \mathbb{E}[X] = \color{blue}{p} \quad var(X) = \color{blue}{p - p^2} \le {1 \over 4} \quad(\text{max variance}) \\ \end{aligned}$$

Binomial with parameter $p \in [0, 1]$

Model number of successes ($k$) in a given number of independent trials ($n$):

$$\begin{aligned} p_X(k) &= \mathbb{P}(X=k) = {n \choose k}p^k(1-p)^{n-k} \\ \mathbb{E}[X] &= n\cdot \color{blue}{p} \quad var(X) = n\cdot\color{blue}{(p - p^2)} \\ \end{aligned}$$

Poisson with parameter $p \in [0, 1]$

Large $n$, small $p$, moderate $\lambda = np$ which is the arrival rate. Model number of arrivals $S$:

$$\begin{aligned} p_S(k) &\to \frac{\lambda^k}{k!}e^{-\lambda} \qquad \mathbf{E}(S) = \lambda \qquad \text{var}(S) = \lambda \\ \end{aligned}$$

Beta with parameters $(\alpha, \beta)$

Infer the posterior unknown bias $\Theta$ of a coin with $k$ number of heads in $n$ (fixed) tosses:

$$\begin{aligned} f_{\Theta|K}(\theta\,|\,k) &= \frac{1}{d(n,k)} \theta^k (1-\theta)^{n-k} \\ \int_0^1 \theta^\alpha (1-\theta)^\beta\,d\theta &= \frac{\alpha! \, \beta!}{(\alpha+\beta+1)!} & \text{beta distribution} \end{aligned}$$

Geometric with parameter $p \in [0, 1]$

Model number of trials ($k$) until a success:

$$\begin{aligned} p_X(k) = \mathbb{P}(X = k) = (1-p)^{k-1}p \quad \mathbb{E}[X] = \frac{1}{p} \quad var(X) = \frac{1-p}{p^2} \\ \end{aligned}$$

Exponential with parameter $\lambda > 0$

Model amount of time elapsed ($x$) until a success:

$$\begin{aligned} f_X(x) &= \lambda e^{-\lambda x} \quad \mathbb{E}[X] = \frac{1}{\lambda} \quad var(X) = \frac{1}{\lambda^2} \\ \mathbb{P}(X > a) &= \int_a^\infty \lambda e^{-\lambda x} \, dx = e^{-\lambda a} \\ \mathbb{P}(T - t > x\, |\, T > t) &= e^{-\lambda x} = \mathbb{P}(T > x) & \text{Memorylessness!} \\ \mathbb{P}(0 \le T \le \delta) &\approx \lambda\delta \approx \mathbb{P}(t \le T \le t+\delta\,|\, T > t) & \mathbb{P}(\text{(success}) \text{ at every }\delta \text{ time step } \approx \lambda\delta\\ \end{aligned}$$

Normal (Gaussian)

$$\begin{aligned} N(0,1): f_X(x) &= \frac{1}{\sqrt{2\pi}} e^{-x^2/2} \\ N(\mu,\sigma^2): f_X(x) &= \frac{1}{\sigma\sqrt{2\pi}} e^{-(x-\mu)^2/2\sigma^2} \\ \end{aligned}$$

$\qquad$If $X \thicksim N(\mu,\sigma^2)$ and $Y = aX + b$, then $Y \thicksim N(a\mu + b,a^2\sigma^2)$

$\qquad$If $X = \Theta + W$ where $W \thicksim N(0,\sigma^2)$, indep. of $\Theta$, then $f_{X|\Theta}(x\,|\,\theta) = f_W(x-\theta)$; or $X \thicksim N(\Theta,\sigma^2)$

Cumulative distributive function (CDF)

Discrete:

$$\begin{aligned} F_X(x) = \mathbb{P}(X \le x) = \sum_{k \le x} p_X(k) \\ \end{aligned}$$

Continuous:

$$\begin{aligned} F_X(x) = \mathbb{P}(X \le x) = \int_{-\infty}^x f_X(t)\,dt \quad \therefore \frac{d}{dx}F_X(x) = f_X(x) \\ \end{aligned}$$

Source: MITx 6.041x, Lecture 5, 6, 8, 14.