Academic writing has changed over time, as I noted in this post back in 2022. The research I referred to in that post identified an increase in the use of adjectives and adverbs over time, noting that as a result, research was becoming less readable over time. The research speculated on the reasons why research had become less readable, but one explanation that they didn't consider was that different generations of academics might express themselves in different ways.
And that is essentially what this 2024 article by Lea-Rachel Kosnik (University of Missouri-St. Louis) and Daniel Hamermesh (University of Texas at Austin), published in the Southern Economic Journal (ungated earlier version here). sets out to look at. Kosnik and Hamermesh look at a sample of all 15,138 articles published in the 'Top 5' economics journals between 1969 and 2018 (the 'Top 5' journals are American Economic Review, Econometrica, Journal of Political Economy, Quarterly Journal of Economics, and Review of Economic Studies). Once they restrict the sample to authors with at least five articles, the sample reduces to 1389 researchers (and 12,812 articles).
Kosnik and Hamermesh then apply sentiment analysis to the articles in the sample, resulting in three scores:
...a positive/negative score (POSN), a certain/tentative score (CERT), and a contemporaneity/past score (CONP).
The POSN score reflects the (positive or negative) emotive tone of the writing, CERT measures how certain or tentative the writing is, and CONP measures whether the writing is contemporary or focused on the past. The measures are normalised by subtracting the average score for all articles in the same field of economics (the same JEL group). Kosnik and Hamermesh look at how these normalised measures vary systematically across the sample of authors and over time, paying particular attention to how the measures are related to the number of years since each researcher completed their PhD. They find that:
Based on the fixed-effects estimates for the entire sample (the 1970s cohort), a one standard-deviation increase in age leads to changes of 0.07 (0.02), -0.03 (-0.01), and -0.05 (-0.03) standard deviations in POSN, CERT, and CONP, respectively.
In other words, older economists write in a more positive emotive tone. However, the effects for CERT and CONP are not statistically significant. Kosnik and Hamermesh then pivot to looking at the square of each normalised measure, contending that it represents the deviation from the norms. It isn't clear to me why they consider that 'more negative' and 'more positive' deviations in norms should be treated identically, and so I don't find that analysis particularly illuminating. It seems like an arbitrary approach, and they note in a footnote that using the absolute value of the normalised measure rather than its square makes the results less statistically significant. That should also give us pause.
However, the basic analysis does provide some other points of interest, including:
Natives write less positively, with less certainty, and with less present/future orientation than do leading economists whose mother tongue is not English. This is true, however, only for those native English-speakers who grew up in North America (57% of authors) or the United Kingdom (5% of authors), whose styles of writing economics are almost identical along the three measures we examine. The styles of the 2% of authors whose native English comes from elsewhere (Ireland, South Africa, Australia, or New Zealand), however, do not differ from those of non-native speakers.
And:
There are also significant differences across the five journals, with all of them being more positive and more contemporary-oriented than the AER, and all but the QJE being written in a more certain voice than the AER...
Clearly, the most dismal scientists get published in the American Economic Review. And:
Additional coauthors, however, do make writing styles more positive, more certain, and less present-oriented, both in the full sample and in the 1970s cohort.
Are sole authors more negative because they have to do all of the work themselves, I wonder? Anyway, Kosnik and Hamermesh then turn to looking at citations, finding that:
While positive deviations of all three measures of sentiment reduce citations significantly or nearly so, the more important question is how large these reductions are. Taking simultaneous one-standard deviation increases in sentiment scores... these increases reduce citations by 5 (2.5)%, or 0.015 (0.01) standard deviations. Writing in a more positive, more certain, or more present-oriented way than others publishing at the same time and in the same sub-field reduces the scholarly impact of one's articles, although the effects are quite small.
Decomposing the change in citations as authors get older, Kosnik and Hamermesh find that:
Scholarly recognition decreases with author's age, but only a small part of the decrease is due to changes in writing style with age.
Finally, Kosnik and Hamermesh look at the subset of Nobel Prize winners, and find that:
Nobelists' style exhibits significantly less certainty than that of other star authors. This example suggests that writing in a more tentative style distinguishes one's scholarship and might provide the scope for subsequent researchers to accord it the attention that helps to generate the distinction of a Nobel Prize.
What do we take away from all this? There is a lot of depth in the analysis, but when we put aside the analyses that rely on the squared measure (which, as I noted above, I don't have as much faith in), it seems that the only remaining result (in terms of age) is that older economists write in a more positive tone than younger economists. Fortunately, the impact of tone on research impact (as measured by citations) is fairly small, so I guess younger economists can afford to be grumpy.
Coming back to where I started this post, what does that imply for changing writing styles over time? If younger economists write in a more negative tone, then as the population (including the population of economists) ages, we might see more positively minded economics writing! Now, the question arises, do younger researchers in other disciplines also write in a more negative tone than older researchers?
[HT: Marginal Revolution, back in 2023]
No comments:
Post a Comment