Probabilistic models that do not interact with each other and have \(\color{blue}{\text{no common sources}}\) of uncertainty.

\[ \begin{aligned} \mathbb{P}(A \cap B) &= \mathbb{P}(A) \cdot \mathbb{P}(B) & \text{iff } A \text{ and } B \text{ are independent}\\ p_{X|A}(x) &= p_X(x) & \text{for all } x \text{ iff } X \text{ and } A \text{ are independent}\\ p_{X,Y}(x,y) &= p_X(x)\cdot p_Y(y) & \text{for all } x,y \text{ iff } X \text{ and } Y \text{ are independent} \\ p_{X,Y,Z}(x,y,z) &= p_X(x)\cdot p_Y(y)\cdot p_Z(z) & \text{for all } x,y,z \text{ iff } X,Y \text{ and } Z \text{ are independent} \\ \end{aligned} \]

Note it's always true that

\[ \begin{aligned} f_{X,Y}(x,y) &= f_{X|Y}(x\,|\,y)\cdot f_Y(y) &\text{by conditional proability}\\ \\ \end{aligned} \]

But

\[ \begin{aligned} f_{X|Y}(x\,|\,y)\cdot f_Y(y) &= f_X(x)\cdot f_Y(y) & \text{iff } X,Y \text{ are independent for all }x,y \\ \end{aligned} \]

Expectation

In general,

\[ \begin{aligned} \mathbb{E}\left[g(x,y)\right] \ne g\big(\mathbb{E}[x], \mathbb{E}[y]\big) \quad \text{eg } \mathbb{E}[XY] \ne \mathbb{E}[X]\mathbb{E}[Y] \end{aligned} \]

It's however always true that

\[ \begin{aligned} \mathbb{E}[aX + b] &= a\mathbb{E}[X] + b & \text{Linearity of Expectation} \end{aligned} \]

But if \(X\) and \(Y\) are independent, then

\[ \begin{aligned} \mathbb{E}[XY] = \mathbb{E}[X]\mathbb{E}[Y] \quad \text{ and } \quad \mathbb{E}\left[g(X)h(Y)\right] = \mathbb{E}\left[g(X)\right]\mathbb{E}\left[h(Y)\right] \end{aligned} \]

Variance

In general,

\[ \begin{aligned} var(X + Y) \ne var(X) + var(Y) \\ \end{aligned} \]

It's however always true that

\[ \begin{aligned} var(aX) = a^2var(X) \quad \text{and} \quad var(X+a) = var(X) \end{aligned} \]

But if \(X\) and \(Y\) are independent, then

\[ \begin{aligned} var(X + Y) = var(X) + var(Y) \\ \end{aligned} \]

Source: MITx 6.041x, Lecture 7.

# posted by rot13(Unafba Pune) @ 8:37 AM