The impact of Sesame Street on education and employment

When I read Andrew Leigh's book Randomistas recently (which I reviewed here), I noted my surprise at the fact that Sesame Street had been subjected to a randomised controlled trial (or rather, more than one). So, when Tim Harford also wrote about Sesame Street last week, it prompted me to follow up. Harford was talking about this article by Melissa Kearney (University of Maryland) and Phillip Levine (Wellesley College), published in the American Economic Journal: Applied Economics (open access, but just in case an ungated version is available here). Long-time readers of this blog might remember my 2014 post about research on the MTV show 16 and Pregnant and its effect on teen pregnancy, which was also based on a paper by Kearney and Levine.

In the Sesame Street paper, they looked at how potential county-level exposure to Sesame Street affected education outcomes in early childhood and high school, and later employment outcomes. They exploit the fact that, despite its popularity, not everyone could watch the show when it first aired:

Following its introduction, Sesame Street was mainly broadcast on PBS channels; of the 192 stations airing the show, 176 of them were affiliated with PBS. The majority of these stations (101) were broadcast on UHF channels rather than VHF channels, which introduced technological constraints that limited exposure to the show. As we detail below, only around two-thirds of the population lived in locations where Sesame Street could be received on their televisions.

Their measure of coverage is based on "distance to the closest television tower broadcasting Sesame Street and whether that tower transmits using UHF or VHF. They then compare cohorts of people who entered first grade between 1959 and 1968 (before Sesame Street first aired) with cohorts who entered first grade from 1970 to 1974, who would have been more likely exposed to Sesame Street before they began school. They then use 1980 Census data to measure 'grade-for-age' (whether people are in the appropriate grade for their age, rather than being held back a year or more), 1990 Census data to measure completed schooling, and 2000 Census data to measure labour market outcomes. When comparing high coverage and low coverage areas, they found that:

...children who were preschool age in 1969 and who lived in areas with greater simulated Sesame Street coverage were statistically significantly more likely to be at the grade level appropriate for their age... A 30-point increase in coverage rates would generate a 3.2 percentage point (0.3 × 0.105 = 0.032) increase in the rate of grade-for-age status. With 20.3 percent of the sample behind their appropriate grade in school, this estimate implies that moving from a weak to strong reception county would lower that rate by around 16 percent...

This effect on grade-for-age status is particularly pronounced among boys. The estimated effect is largest (in absolute terms) for black, non-Hispanic children, but the estimated coefficients are not statistically significantly different across race/ethnic groups.

That effect is comparable in size to contemporary pre-school interventions such as Head Start. The effects are not persistent however, and are mostly gone by high school, where they find:

The results... regarding educational attainment provide no evidence of changes in these outcomes. Parameter estimates are small, statistically insignificant, and inconsistent with the expected pattern across cohorts...

Similarly, using data from the High School and Beyond survey, they find no effect of coverage on educational outcomes. Finally, they find not much of interest in terms of labour market outcomes either:

The results... regarding labor market outcomes suggest some small long-term labor market improvements. Parameter estimates all take on the expected signs: positive for employment and wages, negative for living in poverty. The estimated impact on employment is statistically significant at the 5 percent level.

The magnitude of these effects, though, is small... Our standard for interpreting magnitudes has been to evaluate the impact of moving from a weak reception county to a strong reception county, characterized by a 30 point increase in coverage. These results predict that employment would rise by about 1 percentage point (0.3 × 0.034).

In other words, Sesame Street appears to have good outcomes for young children, ensuring that they maintain grade-for-age status, but those effects are mostly gone by high school or when they are in the labour market later.

However, that might not be the end of this story. As with many studies of this type, migration creates a problem for the analysis. Categorising people's exposure to Sesame Street is necessarily uncertain, particularly if they may have been exposed in one county and then moved to another county that has a different coverage rate. Kearney and Levine try to address this by restricting their analysis to people still living in their state of birth, but migration still creates the potential for measurement error in the most important variable (coverage). Measurement error leads to attenuation bias (a tendency for estimated effects to be biased towards zero in statistical models). This attenuation bias is likely to be least problematic for short term outcomes (like grade-for-age) when not many people have migrated, but will increase over time and be a much larger problem for measuring the effects on labour market outcomes.

So, I don't think we can say for sure that Sesame Street has no long-term effects. If it were at all possible to follow up on some of the participants in the early randomised controlled trials of Sesame Street, we might be able to get a better sense of this. In the meantime, we have only some suggestive evidence of impact beyond the early school years.

Read more:

• Book review: Randomistas