The effect of Netflix on illegal streaming

Consider two goods (Good A and Good B) that are substitutes for each other. Consumers consume one of the goods, or the other. When one of those goods becomes more expensive, some (but not all) consumers will switch to the other good. When one of those goods becomes less available (or unavailable), many (but not all) consumers will switch to the other good.

So, what happens when a movie is no longer available on Netflix? Some consumers will simply watch other movies instead. Other consumers will try to find the movie that they really wanted to watch on some other service. Those other services include illegal streaming. How much will illegal streaming increase when a movie is removed from Netflix? That's essentially the question that this 2023 article by Sarah Frick (UC Berkeley), Deborah Fletcher (Miami University), and Austin Smith (Bates College), published in the Journal of Economic Behavior and Organization (sorry I don't see an ungated version online) tries to answer.

Frick et al. focus on a particular natural experiment:

Epix is an entertainment cable network that features movies and TV shows distributed by Paramount, Lionsgate, and Metro-Goldwyn-Mayer, and its movie content varies from large blockbusters such as The Wolf of Wall Street to smaller independent films... Epix and Netflix upheld an exclusive licensing agreement from 2010 until September 2015 when Netflix announced its decision not to renew the licensing contract with Epix, citing the company’s plan to shift towards hosting its own original content... In response to this, Epix entered into a multi-year agreement with Hulu... Thus, all titles owned by Epix were removed from Netflix on October 1st, 2015 and appeared on Hulu for streaming that same day.

The shift of movies from Netflix to Hulu represents a reduction in availability because:

At the time of the switch, Netflix had roughly 4 times as many subscribers as Hulu, and between July 2015 and December 2016 (the time frame for this study) Google trend searches for “Netflix” were, on average, 5.6 times higher than searches for “Hulu”...

Frick et al. then look at the effect of this change on Google searches of free streaming of each movie that was removed from Netflix as a result of the change. Specifically:

We measure searches in the United States for “watch movie title free online” per month for each movie using Google Ads Keyword Search Planner. This tool provides the absolute number of searches rounded to the nearest tens for values less than 1000 and rounded to the nearest hundreds for values greater than 1000.

...we collect piracy search rates from July 2015 to December 2016 - three months prior to the switch and 15 months after the switch.

They use a difference-in-differences approach, which compares the difference in searches between movies that were, and were not, removed from Netflix, between the time before removal and the time after removal. The control group contains 501 movies that were never removed from Netflix over the period considered (and were also not available on Hulu), while the treatment group includes 141 movies that moved from Netflix only to Hulu only. In this analysis, Frick et al. find that:

...moving Epix movies from Netflix to Hulu results in a 20-22% increase in intent to pirate those movies compared to movies that remained on Netflix. There are distinct heterogeneous effects by movie release year; older movies experience almost three times the increase in piracy following their removal from Netflix compared to newer movies.

Frick et al. go a bit further than that, estimating the cost of illegal streaming:

We calculate that the annual piracy streaming in 2015 for a popular movie in our sample, Hunger Games: Catching Fire was approximately 100 million streams... Assuming each view is linked with at least one search, our 20% result yields an expected 20 million additional piracy searches after this movie was removed from Netflix. Applying estimates from Blackburn, Eisenach, and Harrison (2019) that each illegal viewing displaces 0.14-0.34 paid viewing, the implied impact of removing a movie as popular as Hunger Games: Catching Fire from Netflix is a reduction of 2.8-6.8 million paid viewings annually. To arrive at a dollar cost of these lost viewings for content producers, we multiply these lost views by the $0.41 average revenue per viewing on a streaming platform from Blackburn, Eisenach, and Harrison (ibid.), which yields an average annual lost revenue per movie of $1.15 -$2.79 million.

Given that some 141 movies were moved by Epix from Netflix to Hulu, that may have cost Netflix hundreds of millions of dollars in lost revenue. Of course, there are lots of assumptions embedded in that estimate, not least of which is that subscribers to Netflix don't pay per movie viewing, so actually the marginal revenue to Netflix from one additional viewing is actually zero. The real question is whether losing access to the Epix movies caused Netflix to lose subscribers, since that would be what would really impact their revenue.

So, putting the lost revenue aside since the estimate isn't particularly robust, these results really tell us that when movies are no longer available on Netflix, there is more illegal streaming of those movies. That also implies that having a movie available on Netflix decreases illegal streaming. Which is pretty much exactly how we should expect things to work for substitute goods. 

Challenges in establishing causality in the relationship between alcohol outlets and crime

In my ECONS101 class this week, among other things we discussed the 'faulty causation fallacy'. That occurs when you observe two variables (A and B) that appear to be moving together (either in the same direction or opposite directions), and you assume that a change in Variable A is causing a change in Variable B. You might even be able to tell a really good story about why it is that changes in Variable A cause changes in Variable B. But that doesn't mean that your observation and story about causality is true.

What we observe when we see two variables moving together is correlation. When the two variables move in the same direction, that is positive correlation. When the two variables move in opposite directions, that is negative correlation. [*] Sometimes, when we observe correlation, there really is a causal relationship between the variables. When I push down on the accelerator in my car, my car goes faster. Pushing the accelerator (Variable A) causes a change in the car's speed (Variable B).

However, not all correlations that we observe arise because a change in Variable A causes a change in Variable B. Sometimes, it is the other way around - a change in Variable B causes a change in Variable A. We call this reverse causation. Sometimes, there is some third variable (Variable C), and it is a change in that variable that causes both a change in Variable A and a change in Variable B. We refer to Variable C as a confounder (or a confounding variable). Alternatively, we can say that Variable C is a common cause for both Variable A and Variable B. Finally, the correlation that we observe might be entirely by random chance. In that case, we would say that we have observed a spurious correlation (as in the excellent Tyler Vigen website spurious correlations, which offers up a new classic in the form of correlation #2,204: The number of global pirate attacks is highly correlated with the number of downloads of the Firefox browser - perhaps pirates use Firefox?).

Anyway, I want to illustrate these with an example related to my own research, on the relationship between alcohol outlets and crime. I've published articles on this here and here, with another report here. That research establishes a generally positive correlation between the number (or density) of alcohol outlets and crime. The strength of the correlation varies depending on context - it is different for different locations, and different for different types of alcohol outlets. However, the correlation suggests that where there are more alcohol outlets, there is more crime.

Is this relationship causal though? My earlier research doesn't establish this. However, we can tell a good story, using what is termed availability theory. Availability theory suggests that alcohol consumption depends on the 'full cost' of alcohol - which is made up of the price of alcohol, plus the travel cost of getting to and from the alcohol (such as driving to the alcohol outlet and home). When there are more alcohol outlets in an area, they may compete more vigorously on price, meaning that the first part of the full cost of alcohol is lower. And, when there are more alcohol outlets in an area, consumers don't have to travel as far to get the alcohol, lowering the second part of the full cost of alcohol as well. When there are more alcohol outlets in an area, the full cost of alcohol is lower. And when the full cost of alcohol is lower, people will drink more. And when people drink more, then the amount of crime increases (either because there are more alcohol-impaired victims, or more alcohol-affected offenders). So, this observed relationship could be causal.

On the other hand, there could be reverse causation here. In areas where there is more crime, commercial property rents are lower, and there may be more vacant storefronts. Retailers (including alcohol retailers) looking to set up a store are looking for a vacant storefront, and they will tend to be attracted to low rents. So, perhaps an increase in crime would cause an increase in alcohol outlets, as the crime forces other businesses out of an area?

On the third hand, there could be confounding here. As I noted here, social disorganisation theory is the idea that differences (or changes) in family structures and community stability are a key contributor to differences (or changes) in crime rates between different places (or times). Areas that are more socially disorganised have more crime. Areas that are more socially disorganised are also less able to act collectively to prevent alcohol outlets from opening (or remaining open) in their area. So, social disorganisation might be a confounding variable in the relationship between alcohol outlets and crime, because social disorganisation causes more outlets and more crime.

Finally, the observed relationship could just be a spurious correlation, but spurious correlations tend to arise when you have two variables that are trending over time. In this case, the number of outlets doesn't have an obvious time trend (in some areas it is increasing, and in others it is decreasing), and similarly for crime. So, it seems like there is something more than random chance that leads alcohol outlets and crime to be correlated.

So, we can tell a good story for a causal relationship. However, we can also tell a good story for reverse causation, and a good story for confounding. It requires some careful statistical analysis to disentangle these potential explanations, and that is something that researchers (including myself) will continue to work on. I had an article published in the journal Addiction last year (open access, and I blogged about it here) that shows that at least one potential confounding variable, retail density, doesn't explain the relationship. I also presented at the NZAE Conference a couple of years ago on some further analysis which tentatively suggests that the causal relationship is statistically insignificant (although that research is somewhat hampered by the low quality of alcohol outlets data in New Zealand). There will be more to come on this topic in the future.

*****

[*] This is just one way of conceptualising a correlation between Variable A and Variable B (and I think it is the easiest way). There are other ways we can conceptualise a correlation. For example, if we ignore changes over time, we can observe correlations by looking at variables across different individuals or different areas. In that case, if individuals (or areas) with higher values of Variable A also have higher values of Variable B, that is a positive correlation. And if individuals (or areas) with higher values of Variable A have lower values of Variable B, that is a negative correlation.

Judges are more lenient on defendants' birthdays

Are you nice to people on their birthdays? Probably you are. Most people are. It's a social norm. It turns out that this social norm also extends to judges' decisions about sentencing defendants, as shown in this recent article by Daniel Chen and Arnaud Philippe (both University of Toulouse Capitole), published in the Journal of Economic Behavior and Organization (ungated version here).

Chen and Philippe first look at judicial decisions in France, using data on 4.2 million sentencing decisions over the period from 2003 to 2014. Importantly, in this context:

Judges in correctional courts (for misdemeanor) have no control over their schedule. For each case, when the investigations are finished, the prosecutor in charge chooses the type of procedure (accelerated/normal) and, based on this, picks the next session of the relevant type. The weekly schedule of the sessions is fixed and decided at the beginning of the year by the head of the court with little discretion to select trial dates on defendant birthdays.

So, whether a defendant is sentenced on their birthday or not is effectively random (and Chen and Philippe establish this with some statistical checks in the paper), and which judge the case is assigned to is unrelated to whether it is a defendant's birthday or not. Are judges more lenient on defendants' birthdays? The results are neatly summarised in Figure 2 from the paper:

Notice that the average sentence is substantially lower on a defendant's birthday (the red column) compared to days on either side of their birthday (the blue columns). Statistically:

Results are consistent and indicate that sentences are reduced by roughly four days... On average, sentences are up to 6.2% shorter on defendant birthdays.

So, judges in France are more lenient on defendants' birthdays. Chen and Philippe then turn their attention to the US, where judges have a bit less discretion. As they explain:

Cases are randomly assigned to a single judge. The United States Sentencing Commission (USSC) produces sentencing guidelines for federal judges. The judges are given a guideline range for the criminal sentence that is based on the severity of the crime and the defendant’s criminal history. Due to these guidelines, the largest factor determining sentence range is the criminal charges brought to the judge by the prosecutor. Therefore, we expect the effect of a birthday to be more limited than in France, where judges have more discretion.

Because of the sentencing guidelines, judges have little discretion over the length of the sentence (measured in months), but can vary the number of additional days in the sentence (so, for example, a sentence of 15 months and six days is more lenient than a sentence of 15 months and 20 days). Chen and Philippe therefore focus on differences in the day component of the sentence for US defendants. Their US data is based on over 600,000 sentencing decisions between 1991 and 2003. And their results look very similar to those for France, and are best summarised in Figure 4 from the paper:

Notice again that the red column is much smaller than the blue columns. Statistically:

...the number of days in a federal sentence declines on defendant birthdays, but not on the days before or after birthdays... We find that judges assign 0.13 fewer days if the decision occurs on the defendant’s birthday, all else equal. The effect is about one-third of the average number of days (0.36). We also see no impact on the days before or after the birthday.

So, judges in the US are more lenient on defendants' birthdays. Interestingly, with the US data Chen and Philippe dig a little bit deeper into judicial thinking, since within that data they know which sentences were given by which judges. They also have a dataset of their written judicial decisions. Using those data:

We measure judges’ use of deterrence language and consider it as a proxy for “economic reasoning”... We find that judges below-median in economic thinking are affected by birthdays, decreasing the day component by 0.17, while those above-median in economic thinking are essentially unaffected by birthdays.

Now, if we interpret Chen and Philippe's measure of 'economic reasoning' as a measure of whether judges make decisions in a rational way (in the economic sense), then it appears that judges who are more rational are less affected by the social norm of favouritism on birthdays. That is what we might expect from rational decision-making, which should be based on the costs and benefits of the alternatives (and this applies in sentencing, just as it does in other decision contexts).

Now, Chen and Philippe bury a lot of the detail on this analysis into Appendix C to the paper, but to some extent this is the most interesting of their results. In fact, it would be really interesting to explore this further. Judges' decisions have previously been shown to be affected by whether the decision is made before or after lunch, or affected by weather conditions. It would be interesting to see whether judges who are more rational are less affected by those irrelevant factors as well. There is definitely an opportunity for future research in this area.

The effect of inequality on crime

A rational choice (economic) model of crime would suggest that higher inequality leads to more property crime. This is because, as the disparity between the rich and poor increases, the poor have more incentive to commit property crime, because there is more to gain from such crime, and the opportunity cost of committing crime is lower for the poor than for the rich. Now, this model is easily criticised as unrealistic, as even the relatively wealthy may commit crimes that have an economic motive (Bernie Madoff being the obvious example). The model also doesn't do a good job of explaining violent and other crimes that do not have an obvious economic motive.

Criminologists have a different view of the relationship between inequality and crime. One criminological theory that may be used to explain the relationship is social disorganisation theory. This theory suggests that higher inequality reduces social cohesion, which in turn increases crime - not just property crime, but crime more generally.

Given how easy it is to criticise the economic model of crime, I was interested to read this new article by Matteo Pazzona (Brunel University London), published in the journal World Development (open access). Pazzona conducts a meta-analysis of studies of the relationship between inequality and crime, limiting the analysis to empirical studies in the economics literature (more on that point later). They identified 43 studies, with 1341 estimates of the relationship between inequality and crime (it is not unusual for a study to report multiple estimates, with different covariates and spread across main results and robustness checks). Meta-analysis provides a method of combining those results to estimate an overall effect. In this case, Pazzona finds that:

Firstly, the true values of the partial correlation coefficients – net of publication bias – are statistically but not economically significant. They are in the range 0.007–0.123, which represents non-existent or small effects, according to the guidelines provided by Doucouliagos (2011). Secondly, I also find some limited evidence of positive publication bias (preference for positive results), but its presence is limited.

So, Pazzona concludes at that point in the paper that there is basically no effect of inequality on crime. However, the Doucouliagos paper that he cites says that effects between 0.070 and 0.173 represent a 'small effect', and three of the six point estimates in Pazzona's preferred model fit within this range. So, perhaps there is a small effect of inequality on crime. Which, to be fair to Pazzona, is what he concludes by the end of the paper:

It is safe to say that, if inequality affects crime, its effect is – at best – small.

However, this is clearly not the last word on this topic. Pazzona limits the analysis to include only studies published in the economics literature. That leaves out many studies within the criminological or sociological literature (and possibly other literatures as well). As he notes, three past meta-analyses conducted by criminologists:

...found correlation coefficients higher than the ones found in this research and no evidence of publication bias.

So, that suggests that leaving the criminological literature out of this meta-analysis probably biases the overall effect downwards. Pazzona gives only a very weak rationale for ignoring the studies outside of economics:

By focusing exclusively on economics, I can also limit the large differences in theoretical and methodological approaches with other sciences.

Yes, but at a cost of probably biasing the estimates. We could try to argue that economics has a larger publication bias problem than many other fields (see here), and so the small effect of inequality on crime from the economics literature overall might even over-estimate the true effect. However, Pazzona has very carefully controlled for publication bias in the meta-analysis, 

Coming back to the choice to limit the analysis to economics studies alone, this was an especially inexplicable choice, given that in subsequent analysis in the paper, Pazzona controls for a variety of features of the studies. That analysis could have dealt with the range of methodological approaches that were applied, and actually been helpful in understanding the differences between the findings in the economics literature and those in criminology. In that heterogeneity analysis, Pazzona found that, when looking at the type of crime that was analysed across the 43 studies:

...the coefficient for Property crime is negative and relatively small... The lack of a positive and statistically significant impact on property crime categories implies that inequality does not primarily influence economically motivated criminal behaviour as predicted by the rational choice model.

Score another one against the economists, since the economic model of crime suggests that the effects of inequality on crime should be largest for property crime. How the variables are measured matters, with studies that use crime victimisation survey data reporting larger estimates than those using police data, and using a measure of inequality that is more sensitive to income differences at the bottom of the distribution also increases the estimated relationship with crime. On the latter point, Pazzona notes that:

This provides some evidence that crime incentives are the highest when criminal payoff increases, rather than when the opportunity cost decreases.

I guess, if you believe the economic model of crime, which the other results might give us reason not to. The other variables that are included in a model matter too. Including unemployment and a measure of police deterrence increases the observed effect, while including measures of income or poverty decrease the observed effect. Cross-sectional studies also seem to inflate the observed effect. These results are important, as they show the consequences of methodological choices in the analysis (and, as per my point above, could have helped us understand the differences with the criminology literature).

Overall, this paper is a good case study of how to conduct and report a meta-analysis (and for that reason I have shared it with one of my PhD students who is doing a meta-analysis in quite a different research area). However, the choice to exclude non-economics literature from the analysis leaves the key research question of the relationship between inequality and crime incompletely answered. Clearly, there is more work to do in this area.