Math/Probabilistic

2.6 Conditioning (1)

First-Penguin 2024. 3. 13. 13:08
728x90

Conditional PMFs, given the value of another random variable

Properties of conditional PMFs.

 

Contents

Conditioning (1)

- conditioning a random variable on an event

- conditioning one random variable on another

- summary of facts about conditional PMFs

 

Conditioning (2)

- summary of facts about conditional expectation

- total expectation theorem

+ mean and variance of the geometric (example 2.17.)

+ The Two-Envelopes Paradox (example 2.18. - 이건 따로 다룰 예정)

 

Conditioning a Random Variable on an Event

conditional PMF

The conditional PMF of a random variable \(X\), conditioned on a particular event \(A\) with \(P(A) > 0\), is defined by

$$p_{X|A} (x) = P(X = x|A) = \frac{P({X=x} \cap A)}{P(A)}.$$

 

Note that the events \({X=x} \cap A\) are disjoint for different values of x, their union is A, therefore,

$$P(A) = \sum_x P({X=x} \cap A).$$

 

Combining the above two formulas, we see that $$\sum_x p_{X|A} (x) = 1$$, so \(p_{X|A}\) is a legitimate PMF. (정당한 PMF다~)

 

Figure 2.12: Visualization and calculation of the conditional PMF \(p_{XIA}(x)\). For each X, we add the probabilities of the outcomes in the intersection \({X=x} \cap A\), and normalize by dividing with P(A).

 

 

Conditioning one Random Variable on Another Random Variable

Let X and Y be two random variables associated with the same experiment.

If we know that the value of Y is some particular y [with \(p_Y (y) >0\), this provides partial knowledge about the value of X.

-> conditional PMF \(p_{X|Y}\) of X given Y, which is defined by specializing the definition of \(p_{X|A}\) to events A of the form \({Y=y}\):

$$p_{X|Y} (x|y) = P(X=x|Y=y).$$

 

Using the definition of conditional probabilities, we have

$$p_{X|Y} (x|y) = \frac{P(X=x|Y=y)}{P(Y=y)} = \frac{p_{X,Y} (x,y) }{p_{Y}(y)}.$$

 

 

 

Let us fix some y with \(p_{Y}(y) > 0\) and consider \(p_{X|Y} (x|y)\) as a function of x.

-> a valid PMF for X: it assigns nonnegative values to each possible x, and these values add to 1.

Furthermore, this funciton of x has the same shape as \(p_{X,Y} (x,y)\) except that it is divided by \(p_{Y}(y)\) which enforces the normalization property

$$\sum_x p_{X|Y} (x|y) =1.$$

 

 

 

Counterpart

 

To calculate the marginal PMF

we have by using the definitions,

This formula provides a divide-and-conquer method for calculating marginal PMFs.

It is in essence identical to the total probability theorem given in Chapter 1, but cast in different notation.

 

 

Summary of Facts About Conditional PMFs

Let X and Y be two random variables associated with the same experiment.

  • Conditional PMFs are similar to ordinary PMFs, but pertain to a universe where the conditioning event is known to have occurred.
  • The conditional PMF of X given an event A with P(A) > 0, is defined by $$p_{X|A} (x) = P(X=x|A)$$ and satisfies $$\sum_x p_{X|A}(x) = 1.$$
  • If \(A_1 , . . . ,A_n\) are disjoint events that form a partition of the sample space, with \(P(A_i) > 0 \) for all \(i\), then $$p_{X} = \sum_{i=1}^{n} P(A_i)p_{X|A_i}(x).$$
    (This is a special case of the total probability theorem.)
  • Furthermore, for any event B, with \(P(A_i \cap B) > 0\) for all \(i\), we have $$p_{X|B}(x) = \sum_{i=1}^{n} P(A_i|B)p_{X|A_i \cap B}(x).$$
  • The conditional PMF of \(X\) given \(Y = y\) is related to the joint PMF by $$p_{X, Y} (x, y) = p_Y(y) p_{X|Y}(x|y).$$
  • The conditional PMF of \(X\) given \(Y\) can be used to calculate the marginal PMF of \(X\) through the formula $$p_X(x) = \sum_y p_Y(y) p_{X|Y}(x|y).$$
  • There are natural extensions of the above involving more than two random variables.

 

 

 

 

 

References

Bertsekas, D. P., Tsitsiklis, J. N. (2008). Introduction to Probability Second Edition. Athena Scientific.