Introduction to the Mathematical and Statistical Foundations of Econometrics

Sampling with Replacement

As a third example, consider the quality control example in the previous section except that now the light bulbs are sampled with replacement: After a bulb is tested, it is put back in the stock of N bulbs even if the bulb involved proves to be defective. The rationale for this behavior may be that the customers will at most accept a fraction R/N of defective bulbs and thus will not complain as long as the actual fraction K/N of defective bulbs does not exceed R /N. In other words, why not sell defective light bulbs if doing so is acceptable to the customers?

The sample space ^ and the a-algebra & are the same as in the case of sampling without replacement, but the probability measure P is different. Con­sider again a sample Sj of size n containing k defective light bulbs. Because the light bulbs are put back in the stock after being tested, there are Kk ways of drawing an ordered set of k defective bulbs and (N — K)n-k ways of drawing an ordered set of n — k working bulbs. Thus, the number of ways we can draw, with replacement, an ordered set of n light bulbs containing k defective bulbs is Kk(N — K)n—k. Moreover, as in the Texas lotto case, it follows that the number of unordered sets of k defective bulbs and n — k working bulbs is “n choose k.” Thus, the total number of ways we can choose a sample with replacement containing k defective bulbs and n — k working bulbs in any order is

Kk(N — K)n—k.

Moreover, the number of ways we can choose a sample of size n with replace­ment is Nn. Therefore,

/ n Kk(N — K)n—k V k / Nn

(l)pk (1 — P)n—k ’ k = 0, 1, 2,...,n,

where p = K / N, and again for each set A = {k,..., km }e P (A) =

Xm=i P ({kj}). Of course, if we replace P({k}) in (1.11) by (1.15), the argument in Section 1.2.2 still applies.

The probabilities (1.15) are known as the binomial (n, p) probabilities.

1.2.2. Limits of the Hypergeometric and Binomial Probabilities

Note that if N and K are large relative to n, the hypergeometric probability (1.11) and the binomial probability (1.15) will be almost the same. This follows from

the fact that, for fixed к and n,

[n;K - N + j)] x^;-;(i -§ - n + N + п;.1 (i - N + N)

Рк(1 - P)n к if N — x and K/N — p.

Thus, the binomial probabilities also arise as limits of the hypergeometric prob­abilities.

Moreover, if in the case of the binomial probability (1.15) p is very small and n is very large, the probability (i. i5) can be approximated quite well by the Poisson(k) probability:

ік

P ({к}) = exp(-k)—, к = 0, 1, 2,..., (1.16)

k!

where к = np. This follows from (1.15) by choosing p = k/n for n > k, with к > 0 fixed, and letting n — x while keeping к fixed:

p ({к})=(;у (1 - p )

n!

M(n - к)!

(1 - k/n)n, ^

(1 - k/nj ’к!

because for n - x,

and

(1 - X/n)n ^ exp(-X). (1.17)

Due to the fact that (1.16) is the limit of (1.15) for p = X/n I 0 as n ^to, the Poisson probabilities (1.16) are often used to model the occurrence of rare events.

Note that the sample space corresponding to the Poisson probabilities is ^ = {0, 1,2,...} and that the a-algebra Ж of events involved can be chosen to be the collection of all subsets of ^ because any nonempty subset A of ^ is either countable infinite or finite. If such a subset A is countable infinite, it takes the form A = {k, k2, k3,...}, where the kj’s are distinct nonnegative integers; hence, P(A) = Yj=1 P({kj}) is well-defined. The same applies of course if A is finite: if A = {ki,..., km}, then P(A) = J2’j= P({kj}). This probability measure clearly satisfies the conditions (1.8)-(1.10).