Probabilistic models that do not interact with each other and have $\color{blue}{\text{no common sources}}$ of uncertainty.

$$\begin{aligned} \mathbb{P}(A \cap B) &= \mathbb{P}(A) \cdot \mathbb{P}(B) & \text{iff } A \text{ and } B \text{ are independent}\\ p_{X|A}(x) &= p_X(x) & \text{for all } x \text{ iff } X \text{ and } A \text{ are independent}\\ p_{X,Y}(x,y) &= p_X(x)\cdot p_Y(y) & \text{for all } x,y \text{ iff } X \text{ and } Y \text{ are independent} \\ p_{X,Y,Z}(x,y,z) &= p_X(x)\cdot p_Y(y)\cdot p_Z(z) & \text{for all } x,y,z \text{ iff } X,Y \text{ and } Z \text{ are independent} \\ \end{aligned}$$

Note it's always true that

$$\begin{aligned} f_{X,Y}(x,y) &= f_{X|Y}(x\,|\,y)\cdot f_Y(y) &\text{by conditional proability}\\ \\ \end{aligned}$$

But

$$\begin{aligned} f_{X|Y}(x\,|\,y)\cdot f_Y(y) &= f_X(x)\cdot f_Y(y) & \text{iff } X,Y \text{ are independent for all }x,y \\ \end{aligned}$$

Expectation

In general,

$$\begin{aligned} \mathbb{E}\left[g(x,y)\right] \ne g\big(\mathbb{E}[x], \mathbb{E}[y]\big) \quad \text{eg } \mathbb{E}[XY] \ne \mathbb{E}[X]\mathbb{E}[Y] \end{aligned}$$

It's however always true that

$$\begin{aligned} \mathbb{E}[aX + b] &= a\mathbb{E}[X] + b & \text{Linearity of Expectation} \end{aligned}$$

But if $X$ and $Y$ are independent, then

$$\begin{aligned} \mathbb{E}[XY] = \mathbb{E}[X]\mathbb{E}[Y] \quad \text{ and } \quad \mathbb{E}\left[g(X)h(Y)\right] = \mathbb{E}\left[g(X)\right]\mathbb{E}\left[h(Y)\right] \end{aligned}$$

Variance

In general,

$$\begin{aligned} var(X + Y) \ne var(X) + var(Y) \\ \end{aligned}$$

It's however always true that

$$\begin{aligned} var(aX) = a^2var(X) \quad \text{and} \quad var(X+a) = var(X) \end{aligned}$$

But if $X$ and $Y$ are independent, then

$$\begin{aligned} var(X + Y) = var(X) + var(Y) \\ \end{aligned}$$

Source: MITx 6.041x, Lecture 7.

# posted by rot13(Unafba Pune) @ 8:37 AM