Mostly Harmless Econometrics: An Empiricist’s Companion

The QTE Estimator

The QTE estimator is motivated by the observation that, since the parameters of interest are quantile regression coefficients for compliers, they can (theoretically) be estimated consistently by running quantile regressions in the population of compliers. As always, however, the compliers population is not identifiable; we cannot list the compliers in a given data set. Nevertheless, as in Section 4.5.2, the relevant econometric minimand can be constructed using the Abadie Kappa theorem. Specifically,

where

as before. The QTE estimator is the sample analog of (7.2.3).

There are a number of practical issues that arise when implementing QTE. First, Ki must be estimated and the sampling variance induced by this first-step estimation should be reflected in the relevant asymptotic distribution theory. Abadie, Angrist, and Imbens (2002) derive the limiting distribution of the sample analog of (7.2.3) when Ki is estimated nonparametrically. In practice, however, it is easier to bootstrap the whole procedure (i. e., beginning with the construction of estimated kappas) than to use the asymptotic formulas.

Second, Ki is negative when Di =zi. The kappa-weighted quantile regression minimand is therefore non­convex and no longer has a linear programming representation. This problem can be solved by working with the following minimization problem instead:

min E{E[Ki|Yi, Di, Xi]pT(Yi - aDi - Xib)}

a. b

This minimand is derived by iterating expectations in (7.2.3). The practical difference between (7.2.3) and (7.2.4) is that the term

E [кі |Yi, Di, Xi] — P [Dii > D0i |yL Di, Xi]

is a probability and therefore between zero and one.[108] A further simplification comes from the fact that

Angrist (2001) uses this to implement QTE via a Probit first step to estimate E[Zi|Yi, Di, Xi] separately in the Di — 0 and Di — 1 subsamples, constructing Е[кі|Yi, Di, Xi] using (7.2.5), and then trimming any of the resulting estimates of E[Ki|Yi, Di, Xi] that are outside the unit interval. The resulting first-step estimates of E[Ki|Yi, Di, Xi] can simply be plugged in as weights when constructing quantile regression estimates in a second step using Stata’s qreg command.[109]

Estimates of the Effect of Training on the Quantiles of Trainee Earnings

The Job Training Partnership Act was a large federal program that provided subsidized training to dis­advantaged American workers in the 1980s. JTPA services were delivered at 649 sites, also called Service Delivery Areas (SDAs), located throughout the country. The original study of the labor-market impact of JTPA services was based on 15,981 people for whom continuous data on earnings (from either State unemployment insurance (UI) records or two follow-up surveys) were available for at least 30 months after random assignment.[110] There are 6,102 adult women with 30-month earnings data and 5,102 adult men with 30-month earnings data.

In our notation, Yi is 30-month earnings, Di indicates enrollment for JTPA services, and Zi indicates the randomly assigned offer of JTPA services. A key feature of most social experiments, as with many randomized trials of new drugs and therapies, is that some participants decline the intervention being offered. In the JTPA, those offered services were not compelled to participate in training. Consequently, although the offer of subsidized training was randomly assigned, only about 60 percent of those offered training actually received JTPA services. Treatment received is therefore partly self-selected and likely to be correlated with potential outcomes. On the other hand, the randomized offer of training provides a good instrument for training received since the two are obviously correlated and the offer of treatment is independent of potential
outcomes. Moreover, because of the very low percentage of individuals receiving JTPA services in the control group (less than 2 percent), effects for compliers in this case can be interpreted as effects on those who were treated (there are few always-takers).

Since training offers were randomized in the National JTPA Study, covariates (X;) are not required to consistently estimate effects on compliers. Even in experiments like this, however, it’s customary to control for covariates to correct for chance associations between treatment status and applicant characteristics and to increase precision (see Chapter 2). The covariates used here are baseline measures from the JTPA intake process. They include dummies for black and Hispanic applicants, a dummy for high-school graduates (including GED holders), dummies for married applicants, 5 age-group dummies, and a dummy for whether the applicant worked at least 12 weeks in the year preceding random assignment. Also included are dummies for the original recommended service strategy (classroom, on-the-job training (OJT), job search assistance (JSA), other) and a dummy for whether earnings data are from the second follow-up survey. Since these covariates mostly summarize coarse demographics, we can think of the quantile analysis as telling us how the JTPA experiment affected the earnings distribution within demographic groups.

As a benchmark, OLS and conventional instrumental variables (2SLS) estimates of the impact of training are reported in the first column of Table 7.2.1. The OLS training coefficient is a precisely estimated $3,754. This is the coefficient on D; in a regression of Y; on D; and Xj. These estimates ignore the fact that trainees are self-selected. The 2SLS estimates in Table 7.2.1 use the randomized offer of treatment Z; as an instrument for D;. The 2SLS estimate for men is $1,593 with a standard error of $895, less than half the size of the corresponding OLS estimate.

Quantile regression estimates show that the gap in quantiles by trainee status is much larger (in pro­portionate terms) below the median than above it. This can be seen in the right-hand columns of Table

7.2.1, which reports quantile regression estimates for the.15, .25, .5, .75, and.85 quantiles. Specifically, the.85 quantile of trainee earnings is about 13 percent higher than the corresponding quantile for non-trainees, while the.15 quantile is 136 percent higher. Like the OLS estimates in the table, these quantile regression coefficients do not necessarily have a causal interpretation. Rather they provide a descriptive comparison of the earnings distributions of trainees and non-trainees.

QTE estimates of the effect of training on median earnings are similar in magnitude though less precise than the benchmark 2SLS estimates. On the other hand, the QTE estimates show a pattern very different from the quantile regression estimates, with no evidence of an impact on the.15 or.25 quantile. The estimates at low quantiles are substantially smaller than the corresponding quantile regression estimates, and they are small in absolute terms. For example, the QTE estimate (standard error) of the effect on the.15 quantile is $121 (475), while the corresponding quantile regression estimate is $1,187 (205). Similarly, the QTE estimate (standard error) of the effect on the.25 quantile for men is $702 (670), while the corresponding quantile regression estimate is $2,510 (356). Unlike the results at low quantiles, however, the QTE estimates

of effects on male earnings above the median are large and statistically significant (though still smaller than the corresponding quantile regression estimates).

The result that JTPA training for adult men did not raise the lower quantiles of their earnings is the most interesting finding arising from this analysis. This suggests that the quantile regression estimates in the top half of Table 7.2.1 are contaminated by positive selection bias. One response to this finding might be that few JTPA applicants were very well off, so that distributional effects within applicants are of less concern than the fact that the program helped many applicants overall. However, the upper quantiles of earnings were reasonably high for adults who participated in the National JTPA Study. Increasing earnings in this upper tail is therefore unlikely to have been a high priority.

Notes: The table reports OLS, quantile regression, 2SLS, and QTE estimates of the effect of training on earnings (adapted from Abadie, Angrist, and Imbens (2002)). Assignment status is used as an instrument for training status in Panel B. All models include as covariates dummies for service strategy recommended and age group, and a dummy indicating data from a second follow-up survey. Robust standard errors are reported in parenthesis.