Advanced Econometrics Takeshi Amemiya
Random Variables
At the level of an intermediate textbook, a random variable is defined as a real-valued function over a sample space. But a sample space is not defined precisely, and once a random variable is defined the definition is quickly forgotten and a random variable becomes identified with its probability distribution. This treatment is perfectly satisfactory for most practical applications, but certain advanced theorems can be proved more easily by using the fact that a random variable is indeed a function.
We shall first define a sample space and a probability space. In concrete
terms, a sample space may be regarded as the set of all the possible outcomes of an experiment. Thus, in the experiment of throwing a die, the six faces of the die constitute the sample space; and in the experiment of measuring the height of a randomly chosen student, the set of positive real numbers can be chosen as the sample space. As in the first example, a sample space may be a set of objects other than numbers. A subset of a sample space may be called an event. Thus we speak of the event of an ace turning up or the event of an even number showing in the throw of a die. With each event we associate a real number between 0 and 1 called the probability of the event. When we think of a sample space, we often think of the other two concepts as well: the collection of its subsets (events) and the probabilities attached to the events. The term probability space refers to all three concepts collectively. We shall develop an abstract definition of a probability space in that collective sense.
Given an abstract sample space Q, we want to define the collection A of subsets of Cl that possess certain desired properties.
Definition 3.1.1. The collection A of subsets of £2 is called a o-algebra if it satisfies the properties:
(i) Cl E A. __ __
(ii) EE A=* E E A. (E refers to the complement of E with respect to Q.)
(iii) EjEA, ;'= 1, 2,. . .=> u;_, Ej E A.
Given a ff-algebra, we shall define over it a real-valued set function satisfying certain properties.
Definition 3.1.2. A probability measure, denoted by P( •), is a real-valued set function that is defined over a (7-algebra A and satisfies the properties:
(i) ЕЕА=*Р(Е)Ш0.
(ii) P(Cl)= 1.
(iii) If {Ej) is a countable collection of disjoint sets in A, then
|
|
A probability space and a random variable are defined as follows:
Definition 3.1.3. Given a sample space Cl, a (7-algebra A associated with Q, and a probability measure P{ •) defined over A, we call the triplet (Cl, A, P) a probability space.3
Definition 3.1.4. A random variable on (П, A, P) is a real-valued function4 defined over a sample space Q, denoted by X(co) for соє Q, such that for any real number x,
(coX(co) <x) Є A.
Let us consider two examples of probability space and random variables defined over them. ,
Example 3.1.1. In the sample space consisting of the six faces of a die, all the possible subsets (including the whole space and the null set) constitute a <7-algebra. A probability measure can be defined, for example, by assiging 1/6 to each face and extending probabilities to the other subsets according to the rules given by Definition 3.1.2. An example of a random variable defined over this space is a mapping of the even-numbered faces to one and the odd-numbered faces to zero.
Example 3.1.2. Let a sample space be the closed interval [0, 1]. Consider the smallest ст-algebra containing all the open sets in the interval. Such a сг-algebra is called the collection of Borel sets or a Borel field. This a-algebra can be shown to contain all the countable unions and intersections of open and closed sets. A probability measure of a Borel set can be defined, for example, by assigning to every interval (open, closed, or half-open and half - closed) its length and extending the probabilities to the other Borel sets according to the rules set forth in Definition 3.1.2. Such a measure is called Lebesgue measure.5 In Figure 3.1 three random variables, X, Y, and Z, each of which takes the value 1 or 0 with probability £, are depicted over this probabil-
|
1 1 1
w to a) Figure 3.1 Discrete random variables defined over [0, 1] with Lebesgue measure |
ity space. Note that Z is independent of either X or Y, whereas X and Y are not independent (in fact XY = 0). A continuous random variable X(qj) with the standard normal distribution can be defined over the same probability space by X— Ф-'(&>), where Ф is the standard normal distribution function and 1” denotes the inverse function.
