Happiness is dead?

I've written a number of posts about studies where the main dependent variable of interest is some form of happiness (or, more broadly, subjective wellbeing). There are various ways that subjective wellbeing is measured, but they are all some variant of asking people to rate how satisfied (or happy) they are on a scale, which might be 0-10, 0-5, or might have labelled categories (e.g. very happy; happy; neither happy nor unhappy; unhappy; or very unhappy). Studies then either compare groups of people in terms of the average subjective wellbeing, or investigate the factors that are associated with average subjective wellbeing.

However, it turns out that the whole field of happiness studies might be completely bogus (or, at least, questionable). In a new article published in the Journal of Political Economy (ungated earlier version here), Timothy Bond (Purdue University) and Kevin Lang (Boston University) demonstrate that the main conclusions drawn by basically the entire literature that uses subjective wellbeing are suspect.

Their argument is fairly mathematical:

There are a large (possibly infinite) number of states of happiness that are strictly ranked. In order to calculate a group’s “mean” happiness, these states must be cardinalized, but there are an infinite number of arbitrary cardinalizations, each producing a different set of means. The ranking of the means remains the same for all cardinalizations only if the distribution of happiness states for one group first-order stochastically dominates that for the other.

Essentially, the problem boils down to how you convert a scale like those used to measure subjective wellbeing into a number that can be used for quantitative analysis. It might seem straightforward when the scale is 0-10, but that assumes that the numbers themselves are meaningful. That is, it assumes that the difference between a 2 and a 3 is the same as the difference between a 7 and an 8. It also assumes that every person in the sample rates the scale the same, so that a 7 is the same for everyone. Neither of these assumptions is necessarily true, and things get even worse when you are using a labelled scale rather than a numerical one.

In that case, if you are ranking two groups (A and B) in terms of their average (mean) happiness, and you think that A is the happier group, you can only be sure that you get the same ranking if the rank order of everyone in A is always higher than B. In other words, every percentile of the distribution of A must be happier than the same percentile of the distribution of happiness for B. That's a very strong requirement.

In fact, since this strict requirement is almost never observed in practice, Bond and Lang go on to demonstrate that some of the key results from the happiness literature can be reversed if you make different assumptions about how the scales are converted into numbers (emphasis is mine):

...we never have rank-order identification and can always reverse the standard conclusion by instead assuming a left-skewed or right-skewed lognormal... 

Thus if researchers wanted to draw any conclusions from these data, they would have to eschew rank-order identification. In other words, they would have to argue that it is appropriate to inform policy based on one arbitrary cardinalization of happiness but not on another or, equivalently, that some cardinalizations are “less arbitrary” than others... we further show that nearly every result can be reversed by a lognormal transformation that is no more skewed than the wealth distribution of the United States... Even within this class of distributional assumptions, we cannot draw conclusions stronger than “Nigeria is somewhere between the happiest and least happy country in the world” or “the effect of the unemployment rate on average happiness is somewhere between very positive and very negative.”

Yikes! They conclude that:

It is essentially impossible to rank two groups on the basis of their mean happiness using the types of survey questions prevalent in the literature.

And also:

Certainly calls to replace GDP with measures of national happiness are premature.

It will be interesting to see how pro-happiness researchers respond to this attack on the very foundations of their work.

[HT: Marginal Revolution, back in 2018 when this was still a working paper]