Introduction to the Mathematical and Statistical Foundations of Econometrics

Conditional Probability, Bayes’ Rule, and Independence

1.10.1. Conditional Probability

Consider a statistical experiment with probability space { ^, &, P}, and suppose it is known that the outcome of this experiment is contained in a set B with P (B) > 0. What is the probability of an event A given that the outcome of the experiment is contained in B? For example, roll a dice. Then ^ = {1, 2, 3, 4, 5, 6}, & is the a-algebra of all subsets of ^, and P({«}) = 1/6 for rn = 1,2, 3, 4, 5, 6. Let B be the event The outcome is even (B = {2, 4, 6}), and let A = {1, 2, 3 }. If we know that the outcome is even, then we know that the outcomes {1, 3} in A will not occur; if the outcome is contained in A, it is contained in A П B = {2}. Knowing that the outcome is 2, 4, or 6, the probability that the outcome is contained in A is therefore 1/3 = P(A П B)/P(B). This is the conditional probability of A, given B, denoted by P(A|B). If it is revealed that the outcome of a statistical experiment is contained in a particular set B, then the sample space ^ is reduced to B because we then know that the outcomes in the complement of B will not occur, the a-algebra & is reduced to & П B = {A П B, A є &}, the collection of all intersections of the sets in & with B (Exercise: Is this a a-algebra?), and the probability measure involved becomes P(A|B) = P(A П B)/P(B); hence, the probability space becomes {B, &П B, P(-|B)}. See Exercise 19 for this chapter.