Lock up your daughters and make them watch MTV

Economists are often criticised for using quantitative methods that make little sense to the average person (here is a good example). One of the common tools in the economists toolbox, which is barely used in other disciplines, is the instrumental variables (IV) method. To the average human, IV is almost laughable. Essentially, say you are interested in the effect that one variable (call it 'x0') has on some outcome variable (call it 'y'). Now, instead of running a simple analysis where you see if x0 and y are correlated, you replace x0 with some other variable (call it 'x1', the instrument) and run the analysis to see if x1 is correlated with y. There are good reasons to run an analysis in this way (see here), but I am sympathetic to the ordinary person's critique that this method makes little sense.

To run the IV method, you need two things. First, you need a variable x1, that is a good instrument for x0. For this to be the case, the two variables must be closely correlated - close enough that x1 is a good proxy for x0 (if this isn't the case, then you have a 'weak instrument' problem). This condition isn't often a problem. Second, your variable x1 must not be directly related to y, except through the variable x0. This is the 'exclusion restriction', and I believe this is more often a problem.

Now, an example (maybe!). A few weeks back, this NBER Working Paper (gated, ungated version here) by Melissa Kearney of the University of Maryland and Phillip Levine of Wellesley College hit the news (New York Times story here, CNN here). The authors investigated the effect that the reality TV show 16 and Pregnant had on teenage birth rates in the U.S. The authors conclude:

Our results suggest the introduction of the show led young women to search and tweet about birth control and abortion, indicating that it had some influence on them in a way that could potentially change their behavior. We also find that exposure to the 16 and Pregnant shows had a sizable impact on the rate at which teens give birth in the United States, generating a 5.7 percent reduction in teen teen births that would have been conceived between June 2009, when the show began, and the end of 2010. That can account for roughly one-third of the decline over that period.

First, I though this was a really interesting and worthwhile study. We often hear about the negative impact that media has on behaviour (e.g. violence and television, or video games, food advertising and obesity, and many others). It's good to investigate whether there are positive effects as well.

This study uses the IV method. The reason is explained by the authors:

A critical issue in implementing this approach is accounting for the possibility that locations in which the show is more popular are not randomly selected. Perhaps the show is more popular in locations with elevated rates of teen childbearing. If so, OLS estimates of the relationship between ratings and teen childbearing would include a positive bias, incorrectly suggesting that higher ratings lead to more teen births... To address this form of bias, we utilize an instrumental variables (IV) approach. We instrument for the show’s ratings using ratings among those between ages 12 and 24 for all shows that aired on MTV on weekday evenings between 9:00 and 10:00 in the 4 sweeps months preceding the introduction of 16 and Pregnant (July 2008 through May 2009).

In other words, they replace the television ratings for 16 and Pregnant (x0) with the television ratings for MTV in general from before 16 and Pregnant was introduced (x1). The authors are essentially suggesting that their instrument meets the exclusion restriction, because MTV ratings before 16 and Pregnant began couldn't directly affect birth rates after 16 and Pregnant premiered. But, you ask, if teens are watching more MTV doesn't that leave them with less time to initiate a pregnancy (because they're too busy watching television!)? So, MTV ratings (even excluding 16 and Pregnant) would have an effect on birth rates even without 16 and Pregnant, which violates the exclusion restriction.

By using MTV ratings from before 16 and Pregnant premiered (rather than MTV ratings after), then authors are trying to get around this problem (the argument is that watching more TV before 16 and Pregnant premiered can't directly reduce sexual behaviour afterwards). Here is where I struggle - if MTV ratings after 16 and Pregnant premiered are related to MTV ratings before 16 and Pregnant premiered, and MTV ratings after 16 and Pregnant premiered in turn are related to teen birth rates, then doesn't that violate the exclusion restriction? I think it does.

However, even if you don't believe in the IV approach, the authors have a neat way of convincing us. They show a plausible mechanism through which this causal relationship (16 and Pregnant reducing teen birth rates) would work (see the first sentence of their conclusions above). They show that there is more Google search activity and more Twitter activity relating to 'birth control' immediately following the airing of 16 and Pregnant. The implication is that more searches and mentions of birth control lead to more use of birth control and fewer teen pregnancies.

So, the overall conclusion? If you believe in the IV approach, which many economists do, then 16 and Pregnant caused a significant reduction in teen births. If you don't believe in it, then watching more MTV is related to a significant reduction in teen births. Either of these is an interesting result in its own right.

Closer to home, New Zealand's teenage birth rate peaked at 32.85 per 1000 women aged 15-19 in 2008 and has declined since (to 24.89 in 2012; lower than the 29.4 in the U.S. in 2012). I wonder how many girls in New Zealand are watching 16 and Pregnant? I think I'll advocate that my almost-teenage daughter starts watching more MTV.

[HT: Freakonomics blog]