The measurement of subjective wellbeing (or life satisfaction, or happiness) has attracted a lot of criticism over the last few years (for example, see here and here). The problems arise mostly because we cannot observe people's true happiness, and so instead we use a survey proxy that is typically measured using ordinal categories (for example, very happy, somewhat happy, somewhat unhappy, very unhappy, etc.). Because the way that the proxy measure of happiness maps to 'true happiness' is unknown, researchers who make different distributional assumptions can conclude almost anything. At least, that's the short version of one of the arguments against the current measurement of subjective wellbeing.
However, we may now have a solution of sorts to this problem. As Shuo Liu (Peking University) and Nick Netzer (University of Zurich) explain in this recent article published in the journal American Economic Review (ungated earlier version here), it may be possible to use the length of time a respondent takes to answer the life satisfaction question, to get a measure of the intensity of their happiness (or otherwise). As they explain:
In this paper, we argue that the use of survey response time data can help to solve the problem. Response time is the duration that a survey participant needs to answer a given question. To understand the logic of our argument, consider a happiness survey with just two response categories, “unhappy” and “happy.” Suppose you answer this survey at a moment when you feel very happy. Most likely, you will find it easy to respond “happy” and you will do so quickly. Now suppose you answer the survey at a moment when you feel only moderately satisfied. You may still end up responding “happy” but most likely it will take you longer to decide. The observable distribution of response times among the survey participants who respond to be happy then contains information about the unobservable distribution of happiness within that response category, and analogously for the “unhappy” category. Response time data can provide precisely the evidence that was missing for identification.
Liu and Netzer note that this 'chronometric effect' has been observed in many previous studies, but hasn't previously been applied to the measurement of happiness. They then demonstrate how the use of response times can improve measurement using data from a survey of 8000 MTurk research participants. Specifically, they:
...implemented two versions of the survey, one with two answer categories and one with three answer categories. In both versions of the survey, each substantive question was accompanied by a follow-up question in which participants were asked to refine their previous answer. For example, a subject giving the highest possible response “rather happy” in the initial question about overall life happiness subsequently had the choice between “very happy” and “moderately happy” in the follow-up question.
Conducting the survey online makes it easy to record response times, which we define as the time between the display of the question and the moment when the participant clicked on her answer. To account for individual heterogeneity in response speed, we follow our theoretical analysis and normalize the raw response times by subtracting (in logs) each subject’s response time in the sociodemographic question about marital status, where there are arguably no uncertainties or varying intensities about the correct answer, and which was also answered quickest on average.
Essentially, research participants who were happier should be more certain about being happy, and answer the first happiness question in less time than those who were less certain about being happy. Those happier participants should also be more likely to answer in the follow-up question that they are very happy. And indeed, that is what Liu and Netzer find:
We find that, among subjects who initially gave an identical answer, those who reveal a more extreme position in the follow-up question responded faster on average in the initial question. More specifically, we consider all subjects who responded in the same extreme category in an initial question (e.g., “rather happy”) and partition them into two subgroups based on their response in the follow-up question. Those who give a more extreme response in the follow-up (e.g., “very happy”) should have larger values of the latent variable than those who give a more moderate response (e.g., “moderately happy”). The chronometric effect then predicts that the former should have responded more quickly in the initial question than the latter. We find this prediction confirmed in our data, for both extreme response categories in all seven substantive questions and both versions of the survey.
Liu and Netzer then go on to show similar results when the first question has three levels rather than two levels, although they note that the statistical power is lower in that case.
Overall, these results should provide some comfort for users of subjective wellbeing data, as Liu and Netzer show that the previous concerns about distributional assumptions may be overstated. And, they have provided a way forward, although it is fair to note that this requires that the data be collected digitally (so that response times can be easily captured). Fortunately, it does not require that the data be collected online (which we should be wary of now, as I noted here). So, collection by surveys completed on a tablet or similar should be fit for purpose. Then, either the response time can be used as Liu and Netzer do, or researchers can at least test whether such an adjustment to the underlying subjective wellbeing assessment is necessary.
So, it appears that life satisfaction is not dead. At least, not yet.
Read more:
No comments:
Post a Comment