What happens to the research productivity of winners of top research awards? On the one hand, a research award like a top fellowship or a Nobel Prize might increase a researcher's impact, as other researchers follow the path they have laid down. On the other hand, maybe there is some 'mean reversion', where a previously high-flying researcher simply returns to a less stellar research trajectory (which would look like a decrease in productivity). Or, perhaps a top research award grants a researcher the freedom to explore new, higher-risk areas of research, which could lead to much higher, or much lower, productivity overall?
The question of what happens to researchers after winning a top research award is addressed in this 2023 article by Andrew Nepomuceno, Hilary Bayer, and John Ioannidis (all Stanford University), published in the journal Royal Society Open Science (open access). They looked at the pre- and post-award citation counts for all 72 winners of the Nobel Prize in chemistry, medicine, or physics over the period from 2004 to 2013, and 119 of the 238 McArthur Fellows (only including those in STEM or social science fields) over the same years. Specifically, they compared publications published in two periods of three years: (1) the two years before the award and the year of the award; and (2) the three years after that. They counted citations for the pre-award period up to 2015, and the post-award period up to 2019 (so that both the pre-award and post-award periods had the same number of observed years of citations).
In their main results, Nepomuceno et al. report that:
Nobel Laureates and MacArthur Fellows received fewer citations for post-award work than for pre-award work... The difference was driven predominantly by Nobel Laureates while there was little difference, on average, for pre- versus post-award citation impact for MacArthur Fellows. The median decrease was 80.5 citations among Nobel Laureates and 2 among MacArthur Fellows. For Nobel Laureates, the decrease reached statistical significance (Wilcoxon signed-rank test p = 0.004), whereas for MacArthur Fellows the decrease was not statistically significant (Wilcoxon signed-rank test p = 0.857)...
Post-award citation impact was lower than the pre-award citation impact for 45 of 72 (62.5%) Nobel Laureates and for 63 of the 119 (52.9%) MacArthur Fellows.
Both Nobel Laureates and MacArthur Fellows suffered a reduction in the citation count per-publication after receiving their award, but for different reasons. The Nobel Laureates published the same number of papers in the period after the award as they did before the award. But their lower citations mean that the citation count per-publication was lower. In contrast, the MacArthur Fellows published more papers after the award than they did before the award, but with no change in total citations (again, meaning that the citation count per-publication was lower).
One major difference between the two groups is age - Nobel Laureates are much older than MacArthur Fellows. So, Nepomuceno et al. conducted further analyses stratified by age (in three groups: under 42 years old, 42-57 years old, and over 57 years old), and found that:
...the declining citations pattern was seen only for researchers who were 42 or older at the time of the award, while an opposite pattern was seen for early career researchers who were given an award (especially MacArthur award) at an age of 41 or younger.
However, looking at Table 2 in the paper, it is clear that the negative impact on total citations is largest for the youngest Nobel Prize winners (those aged under 42 years), but is negative for all three age groups. In contrast, there is a positive impact on citations for the youngest McArthur Fellows, and a negative impact for McArthur Fellows aged over 42 years.
Overall, Nepomuceno et al. conclude that:
Although the MacArthur Fellowship and Nobel Prize selection committees share a stated goal of assisting winners in realizing their potential more fully, in terms of citation counts neither the MacArthur Fellowship nor the Nobel Prize heralded increased research impact for the subsequent work and for Nobel Laureates there was even a significant decline.
It is tempting, then, to conclude that these awards are not a good idea. I'm not so sure. I think the research highlights different impacts of the two awards, and I think we learn something potentially important from this. Nobel Laureates may tend to rest on their laurels (pun intended), or may suffer from mean reversion. Or, perhaps they use the profile accorded by their new status as Nobel Laureates to try and have greater policy or political influence, with an opportunity cost of lower research influence. That suggests that it is better to award Nobel Prizes to end-career academics, lest younger academics be diverted from important and path-breaking research. The recent trend in awarding Nobel Prizes to younger recipients (definitely noticeable in economics) may therefore have a negative unintended consequence. In contrast, because there is a positive citation impact for young recipients, the MacArthur Fellowships should be targeted in greater proportion to younger researchers. There is less to be gained from awarding those Fellowships to end-career academics.
To be fair, that is more-or-less how those two awards have historically been allocated: Nobel Prizes to end-career academics, and MacArthur Fellowships ('genius grants') to young stars. This research suggests that might be an important practice to continue.
[HT: Marginal Revolution, back in 2023]
No comments:
Post a Comment