You've probably heard or read about the famous 'marshmallow experiment'. The experimenter leaves a young child alone in a room with a marshmallow, promising that if the child doesn't eat the marshmallow, they will get two marshmallows when the experimenter returns (there are various variations on this test, of course). This 'delayed gratification' test is supposedly predictive of a whole range of later life outcomes, presumably because children who show greater self-control and patience grow into adults who are less impulsive and better at planning, and tend to have various other positive traits associated with good outcomes.
Well, it turns out that the marshmallow test on its own may not be as predictive as originally thought. In this 2020 article published in the Journal of Economic Behavior and Organization (ungated version here), Daniel Benjamin (University of Southern California) and co-authors collected new data on mid-life outcomes for the children who were participants in the original 'Bing' experiment at Stanford in the 1970s. Specifically, they:
...revisit 113 individuals from the original Bing cohort, roughly 45 years after they participated in the original experiments. Within this sample, we examine associations between measures of self-regulation based on multiple assessments during the first four decades of life (including preschool delay) and a comprehensive array of mid-life measures of capital formation. In addition, we also study preschool waiting time on its own as a predictor of mid-life capital formation.
Benjamin et al. used various measures of capital formation in mid-life, including net wealth, permanent income (household income per adult), wealth-income ratio, high interest-rate debt (annual amount of interest paid over 6 percent on any debt), credit card misuse (including having been declined for a credit card, carrying credit card debt from month-to-month, and missing credit card payments), a survey measure of delayed choice, savings rate (as a proportion of income), self-assessed financial health, educational attainment, an index of forward-looking behaviours, and social status (measured on a ten-point scale from lowest status to highest). They pre-registered their analysis, which is probably a good thing given the amount of possibilities here. They focus on two explanatory variables: (1) a rank-normalised measure of self-regulation (RNSRI) based on measures taken at ages 17, 27, and 37, along with the marshmallow test in pre-school; and (2) a rank-normalised measure of delayed gratification from the marshmallow test alone. [*] Based on the first measure, Benjamin et al. find that:
Of the 11 capital formation measures, 10 are associated with the predicted sign. Six variables are significant at our FDR threshold of 0.1 (and the same six have p-values < 0.05): net worth, credit card misuse, financial health, forward-looking behaviors, educational attainment, and permanent income.
Things get interesting when they look at the second measure:
Of the 11 capital formation measures, 6 are positively correlated with RND, but none are significantly associated at our FDR threshold of 0.1 (and all have p-values > 0.05).
Taking those two results together (the broader index explains at least some capital formation variables at mid-life, but the marshmallow test does not), they posit three potential mechanisms:
1. The index of self-regulatory measures is comprised of 86 responses per participant, whereas the preschool delay of gratification task is a single behavioral task. An index of similar measures tends to have a higher signal-to-noise ratio than its components.
2. The preschool delay of gratification task is measured using a diagnostic variant of the task for 34 of our 113 participants; the remaining 79 participants experienced a non-diagnostic variant of the pre-school delay of gratification task. Pooling across diagnostic and non-diagnostic conditions weakens the correlation with outcome variables.
3. The index of self-regulatory measures is comprised of questions that are measured throughout the life course up to age 37 (specifically, ages 4, 17, 27, and 37), whereas the preschool delay of gratification task is measured at age 4. Self- regulation measured closer in time to the observed outcomes will be more strongly related to them.
In their additional analyses, they find support for the first mechanism, some suggestive evidence for the second, and no support for the third. In other words, the marshmallow test isn't very predictive of mid-life outcomes (in terms of capital formation) because it has a low signal-to-noise ratio, as one might expect of a single test conducted once with a pre-schooler. Of course, all of this is just correlations (and always has been, and the researchers are clear about this point), so we shouldn't read too much into it. And perhaps parents can stop stressing about whether their children ate the marshmallows in the test.
Finally, in a sad coda to the Benjamin et al. article, Walter Mischel, the originator of the Bing marshmallow experiments and a co-author on this article, passed away while it was going through the publication process.
[HT: UCLA Anderson Review, via Marginal Revolution]
*****
[*] The rank-normalisation process struck me as rather odd at first, but Benjamin et al. argue that it is necessary to deal with small-sample issues (they only have 113 participants), and in the supplementary material online they present an analysis of the effects of the rank-normalisation process, which seems to suggest that it doesn't bias the results. Still, it is a bit of an oddity to me.
No comments:
Post a Comment