Springer Texts in Business and Economics

Program Evaluation and Difference-in-Differences Estimator

Suppose we want to study the effect of job training programs on earnings. An ideal experiment would assign individuals randomly, by a flip of a coin, to training and non-training camps, and then compare their earnings, holding other factors constant. This is a necessary experiment before the approval of any drug. Patients are randomly assigned to receive the drug or a placebo and the drug is approved or disapproved depending on the difference in the outcome between these two groups. In this case, the FDA is concerned with the drug’s safety and its effectiveness. However, we run into problems in setting this experiment. How can we hold other factors constant? Even twins which have been used in economic studies are not identical and may have different life experiences.

The individual’s prior work experience will affect one’s chances in getting a job after training. But as long as the individuals are randomly assigned, the distribution of work experience is the same in the treatment and control group, i. e., participation in the job training is independent of prior work experience. In this case, omitting previous work experience from the analysis will not cause omitted variable bias in the estimator of the effect of the training program on future employment. Stock and Watson (2003) discuss threats to the internal and external validity of such experiments. The former include: (i) failure to randomize, or (ii) to follow the treatment protocol. These failures can cause bias in estimating the effect of the treatment. The first can happen when individuals are assigned non-randomly to the treatment and non-treatment groups. The second can happen, for example, when some people in the training program do not show up for all training sessions; or when some people who are not supposed to be in the training program are allowed to attend some of these training sessions. Attrition caused by people dropping out of the experiment in either group can cause bias especially if the cause of attrition is related to their acquiring or not acquiring training. In addition, small samples, usually associated with expensive experiments, can affect the precision of the estimates. There can also be experimental effects, brought about by people trying harder simply because the worker being trained feels noticed or because the trainer has a stake in the success of the program. Stock and Watson (2003, p. 380) argue that “threats to external validity compromise the ability to generalize the results of the experiment to other populations and settings. Two such threats are when the experimental sample is not representative of the population of interest and when the treatment being studied is not representative of the treatment that would be implemented more broadly.”

They also warn about “general equilibrium effects”where, for example, turning a small, tem­porary experimental program into a widespread, permanent program might change the economic environment sufficiently that the results of the experiment cannot be generalized. For example, it could displace employer-provided training, thereby reducing the net benefits of the program.

12.7.1 The Difference-in-Differences Estimator

With panel data, observations on the same subjects before and after the training program allow us to estimate the effect of this program on earnings. In simple regression form, assuming the assignment to the training program is random, one regresses the change in earnings before and after training is completed on a dummy variable which takes the value 1 if the individual received training and zero if they did not. This regression computes the average change in earnings for the treatment group before and after the training program and subtracts that from the average change in earnings for the control group. One can include additional regressors which measure the individual characteristics prior to training. Examples are gender, race, education and age of the individual.

Card (1990) used a quasi-experiment to see whether immigration reduces wages. Taking advantage of the “Mariel boatlift” where a large number of Cuban immigrants entered Miami. Card (1990) used the difference-in-differences estimator, comparing the change in wages of low - skilled workers in Miami to the change in wages of similar workers in other comparable U. S. cities over the same period. Card concluded that the influx of Cuban immigrants had a negligible effect on wages of less-skilled workers.

Problems

1. Fixed Effects and the Within Transformation.

(a) Premultiply (12.11) by Q and verify that the transformed equation reduces to (12.12). Show that the new disturbances Qv have zero mean and variance-covariance matrix a2vQ.

Hint: QZЦ = 0.

(b) Show that the GLS estimator is the same as the OLS estimator on this transformed regression equation. Hint: Use one of the necessary and sufficient conditions for GLS to be equivalent to OLS given in Chapter 9.

(c) Using the Frisch-Waugh-Lovell Theorem given in Chapter 7, show that the estimator derived in part (b) is the Within estimator and is given by в = (X'QX)-1X'Qy.

2. Variance-Covariance Matrix of Random Effects.

(a) Show that П given in (12.17) can be written as (12.18).

(b) Show that P and Q are symmetric, idempotent, orthogonal and sum to the identity matrix.

(c) For П-1 given by (12.19), verify that ПП-1 = П-1П = INT.

(d) For П-1/2 given by (12.20), verify that П-1/2П-1/2 = П-1.

3. Fuller and Battese (1974) Transformation. Premultiply y by avQ-1/2 where Q-1/2 is defined in (12.20) and show that the resulting y* has a typical element y*t = yit — 0yi, where the 0 = 1 —av/a1 and a2 = Tj2^ + a)).

4. Unbiased Estimates of the Variance-Components. Using (12.21) and (12.22), show that E(j'2l) = a) and E(J2v) = a2v. Hint: E(u'Qu) = E{tr(u'Qu)} = E{tr(uu'Q)} = tr{E(uu')Q} = tr(QQ).

5.

Swamy and Arora (1972) Estimates of the Variance-Components.

(b) Show that a1 given in (12.26) is unbiased for a

6. System Estimation.

(a) Perform OLS on the system of equations given in (12.27) and show that the resulting esti­mator is Sols = (Z'Z)-1Z'y.

(b) Perform GLS on this system of equations and show that the resulting estimator is Sgls = (Z'Q-1Z)-1Z'Q-1 y where Q-1 is given in (12.19).

7. Random Effects Is More Efficient than Fixed Effects. Using the var(eGLS) expression below (12.30) and varffiwithin) = alwxX, show that

(var(/?GLS))-1 - (var(/Within))-1 = Ф2BxX/jI

which is positive semi-definite. Conclude that var(@Within)- var(eGLS) is positive semi-definite.

8. Maximum Likelihood Estimation of the Random Effects Model.

(a) Using the concentrated likelihood function in (12.34), solve dLc/дф2 = 0 and verify (12.35).

(b) Solve dLc/dj3 = 0 and verify (12.36).

9. Prediction in the Random Effects Model.

(a) For the predictor of yi, T+S given in (12.37), compute E(ui, T+Suit) for t = 1, 2,...,T and verify that w = E(uiT+S u) = a^ffi ® it ) where £i is the г-th column of IN.

(b) Verify (12.39) by showing that (£i ® )P = (ffii ® ).

10. Using the gasoline demand data of Baltagi and Griffin (1983), given on the Springer web site as GASOLINE. DAT, reproduce Tables 12.1 through 12.7.

11. Bounds on s2 in the Random Effects Model. For the random one-way error components model given in (12.1) and (12.2), consider the OLS estimator of var(uit) = a2, which is given by s2 = e'e/(n — K'), where e denotes the vector of OLS residuals, n = NT and K' = K + 1.

(a) Show that E(s2) = a2 + а2^[К'— tr(IN ® JT)PX]/(n — K').

(b) Consider the inequalities given by Kiviet and Kramer (1992) which state that 0 < mean of n — K' smallest roots of Q < E(s2) < mean of n — K' largest roots of Q < tr(Q)/(n — K') where Q = E(uu'). Show that for the one-way error components model, these bounds are

0 < a2 + (n — TK')a2/(n — K') < E(s2) < a2v + na2fl/(n — K') < na2/(n — K').

As n ^ <x>, both bounds tend to a2, and s2 is asymptotically unbiased, irrespective of the particular evolution of X, see Baltagi and Kramer (1994) for a proof of this result.

Verify the relationship between M and M*, i. e., MM* = M*, given below (12.47). Hint: Use the fact that Z = Z* I * with I * = (iN & IK').

Verify that M and M* defined below (12.50) are both symmetric, idempotent and satisfy MM* =

Ml *.

For the gasoline data used in problem 10, verify the Chow-test results given below equation (12.51).

For the gasoline data, compute the Breusch-Pagan, Honda and Standardized LM tests for H0; = °.

If в denotes the LSDV estimator and PGLS denotes the GLS estimator, then

(a) Show that q = eGLS — в satisfies cov(q, PGLS) = 0.

(b) Verify equation (12.56).

For the gasoline data used in problem 10, replicate the Hausman test results given below equation (12.58).

For the cigarette data given as CIGAR. TXT on the Springer web site, reproduce the results given in Table 12.8. See also Baltagi, Griffin and Xiong (2000).

Heteroskedastic Fixed Effects Models. This is based on Baltagi (1996). Consider the fixed effects model

yit ai + Uit i 1, 2,***,N; I 1, 2,---,Ti

where yit denotes output in industry i at time t and a denotes the industry fixed effect. The disturbances uit are assumed to be independent with heteroskedastic variances a2. Note that the data are unbalanced with different number of observations for each industry.

(a) Show that OLS and GLS estimates of a are identical.

(b) Let a2 = MMill Ta2/n where n = ^Mі==1 Ti, be the average disturbance variance. Show that the GLS estimator of a2 is unbiased, whereas the OLS estimator of a2 is biased. Also show that this bias disappears if the data are balanced or the variances are homoskedastic.

(c) Define A2 = a2/a2 for i = 1, 2. ..,N. Show that for a' = (a1, a2,.aN)

^[estimated var(SoLS) — true var(SoLs)]

N

= a2[(n — ^2 A2)/(n — N)] diag (1/Tj) — a2 diag (A2/Tff

i=1

This problem shows that in case there are no regressors in the unbalanced panel data model, fixed effects with heteroskedastic disturbances can be estimated by OLS, but one has to correct the standard errors.

20. The Relative Efficiency of the Between Estimator with Respect to the Within Estimator. This is based on Baltagi (1999). Consider the simple panel data regression model

yit = a + e^it + uit i = 1, 2,...,N; t = 1, 2,...,T (1)

where a and в are scalars. Subtract the mean equation to get rid of the constant yit — У.. = в (xit — x..)+ Uit — U..,

where 1'Ef=1 xit/NT and y.. and u.. are similarly defined. Add and subtract xi from the

regressor in parentheses and rearrange

Vit - У.. = в(xit - Xi.) + в(Хі. - X..) +uit - U.. (3)

where Xi. = 'Ef=1xit/T. Now run the unrestricted least squares regression

Vit - V.. = ew(xit - Xi.) + вb(Xi. - X..) + Uit - U.. (4)

where ew is not necessarily equal to вь-

(a) Show that the least squares estimator of ew from (4) is the Within estimator and that of вь is the Between estimator.

(b) Show that if uit = pi + vit where pi ~ IID(0, a2^) and vit ~ IID(0, a2v) independent of each other and among themselves, then ordinary least squares (OLS) is equivalent to generalized least squares (GLS) on (4).

(c) Show that for model (1), the relative efficiency of the Between estimator with respect to the Within estimator is equal to (BXX/WXX)[(1 - p)/(Tp + (1 - p))], where Wxx = 'EN=1'Ef=1(xit - Xi.)2 denotes the Within variation and Bxx = T'SN=1(Xi. - X..)2 denotes the Between variation. Also, p = a^/(a^ + a2v) denotes the equicorrelation coefficient.

(d) Show that the square of the t-statistic used to test H0; в-w = вь in (4) yields exactly Haus- man’s (1978) specification test.

21. For the crime example of Cornwell and Trumbull (1994) studied in Chapter 11. Use the panel data given as CRIME. DAT on the Springer web site to replicate the Between and Within estimates given in Table 1 of Cornwell and Trumbull (1994). Compute 2SLS and Within-2SLS (2SLS with county dummies) using offense mix and per capita tax revenue as instruments for the probability of arrest and police per capita. Comment on the results.

22. Consider the Arellano and Bond (1991) dynamic employment equation for 140 UK companies over the period 1979-1984. Replicate all the estimation results in Table 4 of Arellano and Bond (1991, p. 290).

References

This chapter is based on Baltagi (2008).

Ahn, S. C. and P. Schmidt (1995), “Efficient Estimation of Models for Dynamic Panel Data,” Journal of Econometrics, 68: 5-27.

Amemiya, T. (1971), “The Estimation of the Variances in a Variance-Components Model,” International Economic Review, 12: 1-13.

Anderson, T. W. and C. Hsiao (1982), “Formulation and Estimation of Dynamic Models Using Panel Data, Journal of Econometrics, 18: 47-82.

Arellano, M. (1989), “A Note on the Anderson-Hsiao Estimator for Panel Data,” Economics Letters, 31: 337-341.

Arellano, M. (1993), “On the Testing of Correlated Effects With Panel Data,” Journal of Econometrics, 59: 87-97.

Arellano, M. and S. Bond (1991), “Some Tests of Specification for Panel Data: Monte Carlo Evidence and An Application to Employment Equations,” Review of Economic Studies, 58: 277-297.

Balestra, P. (1973), “Best Quadratic Unbiased Estimators of the Variance-Covariance Matrix in Normal Regression,” Journal of Econometrics, 2: 17-28.

Baltagi, B. H. (1981), “Pooling: An Experimental Study of Alternative Testing and Estimation Procedures in a Two-Way Errors Components Model,” Journal of Econometrics, 17: 21-49.

Baltagi, B. H. (1996), “Heteroskedastic Fixed Effects Models,” Problem 96.5.1, Econometric Theory, 12: 867.

Baltagi, B. H. (1999), “The Relative Efficiency of the Between Estimator with Respect to the Within Estimator,” Problem 99.4.3, Econometric Theory, 15: 630-631.

Baltagi, B. H. (2008), Econometric Analysis of Panel Data (Wiley: Chichester).

Baltagi, B. H. and J. M. Griffin (1983), “Gasoline Demand in the OECD: An Application of Pooling and Testing Procedures,” European Economic Review, 22: 117-137.

Baltagi, B. H., J. M. Griffin and W. Xiong (2000), “To Pool or Not to Pool: Homogeneous Versus Het­erogeneous Estimators Applied to Cigarette Demand,” Review of Economics and Statistics, 82: 117-126.

Baltagi, B. H. and W. Kramer (1994), “Consistency, Asymptotic Unbiasedness and Bounds on the Bias of s2 in the Linear Regression Model with Error Components Disturbances,” Statistical Papers, 35: 323-328.

Breusch, T. S. (1987), “Maximum Likelihood Estimation of Random Effects Models,” Journal of Econo­metrics, 36: 383-389.

Breusch, T. S. and A. R. Pagan (1980), “The Lagrange Multiplier Test and its Applications to Model Specification in Econometrics,” Review of Economic Studies, 47: 239-253.

Card (1990), “The Impact of the Mariel Boat Lift on the Miami Labor Market,” Industrial and Labor Relations Review, 43: 245-253.

Chow, G. C. (1960), “Tests of Equality Between Sets of Coefficients in Two Linear Regressions,” Econo - metrica, 28: 591-605.

Cornwell, C. and W. N. Trumbull (1994), “Estimating the Economic Model of Crime with Panel Data,” Review of Economics and Statistics 76: 360-366.

Evans, M. A. and M. L. King (1985), “Critical Value Approximations for Tests of Linear Regression Disturbances,” Australian Journal of Statistics, 27: 68-83.

Fisher, F. M. (1970), “Tests of Equality Between Sets of Coefficients in Two Linear Regressions: An Expository Note,” Econometrica, 38: 361-366.

Fuller, W. A. and G. E. Battese (1974), “Estimation of Linear Models with Cross-Error Structure,” Jour­nal of Econometrics, 2: 67-78.

Goldberger, A. S. (1962), “Best Linear Unbiased Prediction in the Generalized Linear Regression Model,” Journal of the American Statistical Association, 57: 369-375.

Graybill, F. A. (1961), An Introduction to Linear Statistical Models (McGraw-Hill: New York).

Hansen, L. P. (1982), “Large Sample Properties of Generalized Method of Moments Estimators,” Econo - metrica, 50: 1029-1054.

Hausman, J. A. (1978), “Specification Tests in Econometrics,” Econometrica, 46: 1251-1271.

Honda, Y. (1985), “Testing the Error Components Model with Non-Normal Disturbances,” Review of Economic Studies, 52: 681-690.

Hsiao, C. (2003), Analysis of Panel Data (Cambridge University Press: Cambridge).

Judge, G. G., W. E. Griffiths, R. C. Hill, H. Lutkepohl and T. C. Lee (1985), The Theory and Practice of Econometrics (Wiley: New York).

Kiviet, J. F. and W. Kramer (1992), “Bias of s2 in the Linear Regression Model with Correlated Errors,” Empirical Economics, 16: 375-377.

Maddala, G. S. (1971), “The Use of Variance Components Models in Pooling Cross Section and Time Series Data,” Econometrica, 39: 341-358.

Maddala, G. S. and T. Mount (1973), “A Comparative Study of Alternative Estimators for Variance Components Models Used in Econometric Applications,” Journal of the American Statistical As­sociation, 68: 324-328.

Moulton, B. R. and W. C. Randolph (1989), “Alternative Tests of the Error Components Model,” Econo - metrica, 57: 685-693.

Nerlove, M. (1971), “A Note on Error Components Models,” Econometrica, 39: 383-396.

Nickell, S. (1981), “Biases in Dynamic Models with Fixed Effects,”Econometrica, 49: 1417-1426.

Searle, S. R. (1971), Linear Models (Wiley: New York).

Sargan, J. (1958), “The Estimation of Economic Relationships Using Instrumental Variables,” Econo - metrica, 26: 393-415.

Swamy, P. A.V. B. and S. S. Arora (1972), “The Exact Finite Sample Properties of the Estimators of Coefficients in the Error Components Regression Models,” Econometrica, 40: 261-275.

Taub, A. J. (1979), “Prediction in the Context of the Variance-Components Model,” Journal of Econo­metrics, 10: 103-108.

Taylor, W. E. (1980), “Small Sample Considerations in Estimation from Panel Data,” Journal of Econo­metrics, 13: 203-223.

Wallace, T. and A. Hussain (1969), “The Use of Error Components Models in Combining Cross-Section and Time-Series Data,” Econometrica, 37: 55-72.

Wansbeek, T. J. and A. Kapteyn (1978), “The Separation of Individual Variation and Systematic Change in the Analysis of Panel Data,” Annales de l’INSEE, 30-31: 659-680.

Wansbeek, T. J. and A. Kapteyn (1982), “A Simple Way to Obtain the Spectral Decomposition of Variance Components Models for Balanced Data,” Communications in Statistics All, 2105-2112.

Wansbeek, T. J. and A. Kapteyn, (1989), “Estimation of the error components model with incomplete panels,” Journal of Econometrics 41: 341-361.

Zellner, A. (1962), “An Efficient Method of Estimating Seemingly Unrelated Regression and Tests for Aggregation Bias,” Journal of the American Statistical Association, 57: 348-368.

CHAPTER 13