Book review: Measuring What Counts

I've read a number of books that critique economics, or set up a strawman version of economics as a target as part of a critique of capitalism or market fundamentalism - for example, Doughnut Economics (which I reviewed here) or What Money Can't Buy (which I reviewed here). The problem with those books is that the strawman argument detracts from what might otherwise be a reasoned critique of current economic structures. Fortunately, that isn't the case for Measuring What Counts, by Joseph Stiglitz, Jean-Paul Fitoussi, and Martine Durand. This is a book written by well-respected economists, so its critique of GDP as a measure of wellbeing (which is the core of the book) is worth listening to.

However, let me take a step back. The origins of this book date back to the 2008 Commission on the Measurement of Economic Performance and Social Progress, instituted by French President Nicolas Sarkozy, and chaired by Joseph Stiglitz, with Amartya Sen and Jean-Paul Fitoussi. The purpose of the so-called Stiglitz-Sen-Fitoussi Commission was to reassess the adequacy of current metrics of performance and progress, and to suggest alternative approaches. The Commission's 292-page final report was released in 2009. The OECD followed that report up by convening a High-Level Expert Group (HLEG), chaired by Stiglitz, Fitoussi, and Durand. Its final reports were released in 2018. This book is a summary of those reports, written by the three co-chairs.

The book has a common theme that runs throughout, as summarised in the Overview section:

Its central message is that what we measure affects what we do. If we measure the wrong thing, we will do the wrong thing. If we don't measure something, it becomes neglected, as if the problem didn't exist.

The critique of GDP that Stiglitz, Fitoussi, and Durand present is well-crafted, but there is little new in there. However, in brief you can think of that as being an expansion on the idea of Goodhart's Law: That when a measure becomes a target, it ceases to be a good measure. Since GDP (and economic growth as measured by growth in GDP) has become a target of governments, with high growth seen as good and low growth (or negative growth) seen as bad, GDP has ceased to be a good measure of human wellbeing or progress. The truth, of course, is that GDP was never a measure of human wellbeing or progress, but a measure of production.

Stiglitz, Fitoussi, and Durand make the case that, rather than focusing on a single measure of progress (GDP), it would be better for governments to have a 'dashboard' of indicators covering all manner of indicators across many domains of human wellbeing. One example they present is the Sustainable Development Goals. However, they caution:

...the demand for comprehensiveness of the SDGs had an adverse effect: some 17 goals, with 169 targets and 232 indicators, were eventually listed - too many to be meaningfully comprehended or to be a focus of policy.

So, one of the things we are to take from this book is that a single measure (GDP) with a single goal (economic growth) is not enough, while 232 indicators associated with 17 goals is too many. However, we never get a sense of where the 'Goldilocks zone' is for the number of measures or goals that a government should employ. I felt that was one of the let-downs of the book. The problem with the dashboard approach is that having a large number of indicators can easily lead a policymaker to justify any policy on the basis of its effect on one of the indicators, or to trumpet any policy as a success because of its effects on one of the indicators, while ignoring or downplaying its effects on other indicators. To be fair, Stiglitz, Fitoussi, and Durand are not oblivious to these critiques. However, the book lacks a level of criticality, which surprises me somewhat given the previous books I have read by Stiglitz in particular. A broader section devoted to the challenges in this approach would have been a welcome addition.

Despite the obvious critiques of the dashboard approach, it does have the potential for making the trade-offs inherent in future policy, and in the evaluation of current and past policy, more transparent (although establishing the counterfactual will always present a tricky problem for evaluation). However, that relies on policymakers, politicians, the media, and the public being able to interpret the dashboard in terms of trade-offs. This was another thing that I felt was missing from the book.

Nevertheless, the book does a good job of summarising where things stand (as at 2019) with what the authors term the 'Beyond GDP' movement. Some governments are much further along than others. As one example, which the authors brief mention, consider the New Zealand Treasury's Living Standards Framework Dashboard. If you haven't already tried this out, you should. It is an excellent source of summary information on wellbeing for New Zealand.

Finally, the book provides some pointers to examples of current best practice, as well as an outline of future research and data needs. On that last point, in particular:

...economic insecurity, inequality of opportunity, trust, and resilience - currently lack a foundation in countries' statistical system. Greater investment is needed in all these areas, as the arguable lack of more-adequate metrics contributed to inadequate policy decisions.

As a summary, the book lacks some of the detail necessary to fully understand some of the points that were made. Fortunately, there is a companion edited volume, titled For Good Measure. I've ordered that one, and look forward to bring you a review of it soon. I enjoyed this book, and it does a far better job of critiquing GDP and presenting an alternative that is worth further exploration. And it didn't require a doughnut metaphor to do so.