Corporate prediction markets work well, given time

Prediction markets were all the rage in the 2000s. The Iowa Electronic Markets were at their peak, and James Surowiecki wrote the bestseller The Wisdom of Crowds. The basic idea is that the average forecast of a bunch of people is better than the forecast of most (or sometimes all) experts. I was quite surprised that prediction markets appeared to go away in recent years, or at least they weren't in the news much (probably crowded out by stories about big data). I was especially surprised we didn't see many stories about corporate prediction markets, which suggested they weren't particularly prevalent. It turns out that wasn't the case at all.

This paper in the Review of Economic Studies (ungated earlier version here) by Bo Cowgill (UC Berkeley) and Eric Zitzewitz (Dartmouth College) shows that this wasn't the case at all, as demonstrated by their Table 1:

The paper has much more of interest of course. It looks at three prediction markets, at Google, Ford, and an unnamed basic material and energy conglomerate (Firm X), and tests whether these markets are efficient. They find:

Despite large differences in market design, operation, participation, and incentives, we find that prediction market prices at our three companies are well calibrated to probabilities and improve upon alternative forecasting methods. Ford employs experts to forecast weekly vehicle sales, and we show that contemporaneous prediction market forecasts outperform the expert forecast, achieving a 25% lower mean-squared error... At both Google and Firm X market-based forecasts outperform those used in designing the securities, using market prices from the first 24 hours of trading so that we are again comparing forecasts of roughly similar vintage.

In other words, the prediction markets perform well. There are some inefficiencies though - for instance, Google's market exhibits an optimism bias, which is driven by traders who are overly optimistic about their own projects (and their friends' projects), as well as new hires being especially optimistic. However, the inefficiencies disappear over time, and:

Improvement over time is driven by two mechanisms: first, more experienced traders trade against the identified inefficiencies and earn higher returns, suggesting that traders become better calibrated with experience. Secondly, traders (of a given experience level) with higher past returns earn higher future returns, trade against identified inefficiencies, and trade more in the future. These results together suggest that traders differ in their skill levels, they learn about their ability over time, and self-selection causes the average skill level in the market to rise over time.

So, prediction markets work well for firms, given enough time for inefficiencies to be driven out of the markets. This is what you would expect - traders who are consistently poor forecasters either drop out of the market or they learn to be better forecasters.

However, as the authors note they were limited to looking at just three prediction markets, and only those who would share data with them. There is likely to be some survival bias here - prediction markets that don't work well won't last in the market long, and are unlikely to be observed. On the other hand, by the time of writing the paper, the markets at Google and Ford had closed down in spite of their good overall predictive performance. On this last point, the authors note that "decisions about the adoption of corporate prediction markets may... depend on factors other than their utility in aggregating information". Other forecasters don't like being shown up.

[HT: Marginal Revolution]