The irrationality of NFL play callers, part 2

A few weeks ago I wrote a post about the irrationality of NFL offensive play callers, specifically that they fail to adequately randomise their play choices, with the implication that defensive play callers should be able to (and do) exploit this for their own gain. Why would they do this? The Emara et al. paper (one of the two I used in the post) suggested:

Perhaps teams feel pressure not to repeat the play type on offense, in order to avoid criticism for being too “predictable” by fans, media, or executives who have difficulty detecting whether outcomes of a sequence are statistically independent. Further, perhaps this concern is sufficiently important so that teams accept the negative consequences that arise from the risk that the defense can detect a pattern in their mixing.

Which seems like a plausible suggestion. Last week I read this recent post by Jesse Galef on the same topic:

In football, it pays to be unpredictable (although the “wrong way touchdown” might be taking it a bit far.) If the other team picks up on an unintended pattern in your play calling, they can take advantage of it and adjust their strategy to counter yours. Coaches and their staff of coordinators are paid millions of dollars to call plays that maximize their team’s talent and exploit their opponent’s weaknesses.

That’s why it surprised Brian Burke, formerly of AdvancedNFLAnalytics.com (and now hired by ESPN) to see a peculiar trend: football teams seem to rush a remarkably high percent on 2nd and 10 compared to 2nd and 9 or 11.

What’s causing that?

Galef argues that there are two possibilities (note that the first one is similar to the suggestion by Emara et al.):

1. Coaches (like all humans) are bad at generating random sequences, and have a tendency to alternate too much when they’re trying to be genuinely random. Since 2nd and 10 is most likely the result of a 1st down pass, alternating would produce a high percent of 2nd down rushes.

2. Coaches are suffering from the ‘small sample fallacy’ and ‘recency bias’, overreacting to the result of the previous play. Since 2nd and 10 not only likely follows a pass, but a failed pass, coaches have an impulse to try the alternative without realizing they’re being predictable.

Galef then goes through some fairly pointy-headed methodological stuff, before arriving at his conclusion:

If their teams don’t get very far on 1st down, coaches are inclined to change their play call on 2nd down. But as a team gains more yards on 1st down, coaches are less and less inclined to switch. If the team got six yards, coaches rush about 57% of the time on 2nd down regardless of whether they ran or passed last play. And it actually reverses if you go beyond that – if the team gained more than six yards on 1st down, coaches have a tendency to repeat whatever just succeeded.

It sure looks like coaches are reacting to the previous play in a predictable Win-Stay Lose-Shift pattern...

All signs point to the recency bias being the primary culprit.

However, I'd still like to see some consideration of risk aversion here. Galef controlled for game situation and a bunch of other game- and team-level variables, but not individual-level variables related to the coaches (he did control for quarterback accuracy, but as far as I can see that might be a team-level variable if the team has changed quarterback mid-season).

This is yet more evidence that there is an exploitable trend in NFL offensive play calling, but the reason underlying this trend is still not fully established. Defensive play callers need not care about the reasons why though - they should be adjusting their strategies now.

Read more:

• The irrationality of NFL play-callers