The tone and expression of academics on X (or Twitter)

In my previous post, I highlighted the apparent contribution of X (formerly Twitter) to toxicity on the Economics Job Market Rumors (EJMR) website. A natural follow-up question is whether and to what extent academics on X contribute to the toxicity on that platform and, by extension, to other forums such as EJMR. This recent article by Prashant Garg (Imperial College London) and Thiemo Fetzer (University of Warwick), published in the journal Nature Human Behaviour (open access), goes some way towards providing an answer.

Garg and Fetzer constructed a dataset of nearly 100,000 academics, including all of their Twitter [*] activity from 2016 to 2022. They then use large language models (ChatGPT-3.5 and GPT-4) to characterise each tweet in relation to content and tone. They assess each academic's stance on climate change, economic policy, and cultural issues. In terms of tone, they measure egocentrism (how often the academic refers to themselves in the first person), toxicity (based on the probability a tweet is classified as toxic by Google's Perspective API), and the balance between reason and emotion (measured as a ratio of 'affective terms' to 'cognitive terms' based on the Linguistic Inquiry and Word Count tool). The analysis is then largely descriptive, but nonetheless interesting.

Garg and Fetzer first find that:

...leading academics are not typically social media influencers... We found weak correlations between citation counts and Twitter metrics: citations and likes... citations and followers... and citations and content creation...

Garg and Fetzer observe that:

The weak correlation underscores that many prominent public intellectuals online gain visibility through public engagement rather than scholarly achievements, often holding lower academic credentials while commanding significant public attention, thus widening the gap between social media influencers and established academic experts.

I think that Garg and Fetzer overstate the case here. The weak correlations suggest that Twitter includes a cross-section of academics (in terms of academic quality), rather than that the top academics eschew Twitter (which would instead lead to negative correlations between measures of academic quality and Twitter engagement).

I'll put aside their results on political expression, which I round rather uninteresting. In contrast, the results in terms of tone demonstrate some interesting correlations. First, in terms of egocentrism (using self-referential terms such as 'I', 'me', 'my', and 'myself'):

Female academics... exhibit higher egocentrism than male academics...

Egocentrism increases with university ranking: academics at top-100 institutions... exhibit higher egocentrism than those from institutions ranked 101-500... US-based academics... show higher egocentrism than non-US academics

Then, in terms of toxicity:

Academics with high reach but low academic credibility... exhibit lower toxicity than those with the contrasting profile, that is, ones with low reach but high credibility...

Academics at top-100 universities... exhibit higher toxicity than those at institutions ranked 101-500... Moreover, US-based academics... exhibit higher toxicity than non-US academics...

And in terms of emotionality (or reason):

Emotionality is significantly higher among female academics... than male academics... In terms of reach and credibility, high-reach/low-credibility scholars... show significantly higher emotionality than low-reach/high-credibility scholars...

Finally, US-based academics... exhibit higher emotionality than non-US scholars...

Many of those differences will surprise no one, such as US-based academics being more egocentric and toxic in their expression on Twitter. Other differences seem to confirm familiar stereotypes, such as female academics using more emotional language than male academics. No doubt, some of the differences relate to differences in norms across different disciplines in terms of communication styles (both on Twitter and in general academic discourse). Garg and Fetzer don't control those other factors that might affect tone and expression. And before we get carried away about how toxic academics are on Twitter, Garg and Fetzer provide an important comparison with the general population. From Figure 6 in the paper:

Notice that academics (the blue line) exhibit far less toxicity (in the graph in the top middle) than the general population of Twitter users (the red line). Moreover, the trend in toxicity is downwards (for academics over the whole period from 2016 to 2023, and for the general population from 2021 to 2023). So, academics are not the main problem in terms of toxicity in the discourse on Twitter.

Nevertheless, there are important differences across academics, and one difference in particular stands out. Academics with high reach (those that are very active on Twitter) but low academic credibility (they are not highly credible academics, as measured by citations) exhibit less toxic expression on Twitter than other academics, particularly those who have low reach but high academic credibility. In their conclusion, Garg and Fetzer focus on this as a problem because:

...those with the greatest public reach may not represent top scholars, potentially distorting public perceptions

However, I see the opposite problem. In terms of tone, the top scholars with the lowest reach have the most toxic expression. Are those the sorts of academics that we want to promote even further on social media? I would suggest not.

What is a better option? First, more highly credible academics should be encouraged to engage in the social media discourse. However, it is important to recognise that credibility alone is not enough. What is needed are credible academics who also model constructive discourse without the toxicity, raising the standard of debate. However, as noted in yesterday's post, many high-quality (especially female) scholars are targets of hostility on social media. These are not separate issues.

Alternatively, we could raise the standard of academic discourse on Twitter more generally, without changing who is represented on the platform. That would reduce the toxic nature of the interactions. Stop laughing! It could happen. The tone and expression of academics on X (or Twitter) matters. Academics can set the standards for everyone else. We don't need to descend into the toxic culture wars that play out each day on social media. We are better than that, and if we show ourselves to be such, maybe more people will listen.

[HT: Marginal Revolution, last year]

*****

[*] I refer to the platform mostly as Twitter, because it didn't change names to X until July 2023, after Garg and Fetzer's dataset ends.

More on the toxic environment in Economics Job Market Rumors

The Economics Job Market Rumors (EJMR) website began as a forum for PhD students to discuss the economics job market, but it has long since become notorious for misogyny, racism, and other toxic behaviour (see this post, for example), due in large part to the anonymous nature of the platform. And even though the user community at EJMR has been called out for their behaviour, it doesn't seem to have gotten much better over time. This is documented by this 2025 article by Florian Ederer (Boston University), Paul Goldsmith-Pinkham, and Kyle Jensen (both Yale University), published in the journal AEA Papers and Proceedings (ungated earlier version here).

Ederer et al. analyse content from EJMR over the period from January 2012 to May 2023, documenting a number of changes. First:

...starting in 2018, EJMR saw an explosion in discussions initiated by references to Twitter posts. This shift mirrors Twitter’s growing importance as a real-time source of information and debate in academic and public policy circles.

Twitter (now X) essentially took over from YouTube as being the source of initial references on EJMR from about 2018, which is about the time of the earlier research on toxicity and misogyny on the platform. There were also surprising declines in Marginal Revolution and NBER links as the starting point for EJMR discussions. Given the predominance of Twitter as a source, Ederer et al. then look in more detail at which Twitter accounts were most referenced, reporting that:

These accounts can be broadly categorized into three main groups: economists, right-wing commentators, and journalists. The group of economists (e.g., Claudia_Sahm, jenniferdoleac, and JustinWolfers) includes academic and professional economists from diverse institutions whose tweets often serve as springboards for debates on research findings, policy implications, and professional conduct. The second group includes polarizing and predominantly conservative commentators and agitators (e.g., realChrisBrunet, RichardHanania, and libsoftiktok) and reflects EJMR’s right-wing slant and engagement with contentious political and social issues. The third group is a collection of news sources and journalistic accounts, many of which have a conservative slant (e.g., visegrad24, disclosetv, and nypost).

Finally, Ederer et al. characterise the posts linking to each Twitter account in terms of 'hate speech', 'negativity', 'misogyny', and 'toxicity' (based on measures from their companion paper here), finding that:

Among the 10 most frequently mentioned Twitter accounts, there are four economists, including three female economists. EJMR posts referencing two of these female economists (Claudia_Sahm and jenniferdoleac) have very high average z-scores of 1.974 and 2.598 for the Misogynistic classifier, indicating that EJMR posters discuss them in strongly misogynistic terms compared to all other Twitter accounts mentioned on EJMR... The only other large average z-score for the Misogynistic measure is for EJMR posts referencing elben (z-score Misogynistic = 0.956), an academic economist who has championed LGBTQ-inclusive policies in the economics profession.

In other words, since 2018 EJMR has remained a hostile and misogynistic platform, with its toxicity increasingly fed by same antagonism and culture-war discourse on Twitter/X. EJMR is not just an academic forum, but has become part of that broader hostile ecosystem.

Economists need places where they can share research in progress, ideas, and practical advice, especially early in their careers. In its early days, EJMR served that purpose. However, it has long since become a space that early career economists are better off avoiding.

[HT: Marginal Revolution, in January last year]

Read more:

• The toxic environment for women in econjobrumors.com

This week in research #119

Here's what caught my eye in research over the past week (another very quiet week, it seems):

• Clemens et al. analyse the effect of California's $20 fast food minimum wage, which was implemented in 2024, and find that food away from home prices increased by 3.3 to 3.6 percent in areas subject to the minimum wage relative to control areas (so firms passed on their cost increase to consumers)

Evidence that artificial intelligence is increasing the impact, but narrowing the scope, of research

There is growing evidence of positive impacts of generative artificial intelligence on productivity. This includes productivity in research (see this post, for example), including my own. However, some have questioned whether increasing research productivity comes at a cost of narrowing the scope of research.

So, I was interested to read this article by Qianyue Hao (Tsinghua University) and co-authors, published in the prestigious journal Nature (ungated earlier version here) late last year. They look at the impact of AI tools (not limited to generative AI) on the productivity of researchers and the quality of research. Specifically, they look at authors publishing in six representative fields: biology, medicine, chemistry, physics, materials science, and geology, across three 'eras': (1) the 'machine learning era ' (from 1980 to 2014), the 'deep learning era' (from 2015 to 2022), and the 'generative AI era' (from 2023 onwards). Hao et al. compare authors who publish 'AI augmented papers' with those who do not. An 'AI augmented paper' is one that uses methods such as:

...support vector machines and principal component analysis from the machine learning era, and convolutional neural networks and generative adversarial networks from the deep learning era. Large language models, which have emerged in recent years, also rank among the most frequently used methods...

Using a dataset that includes over 27 million papers with complete records that were published between 1980 and 2025, of which about 310,000 were 'AI augmented', Hao et al. find that:

...annual citations to AI papers are 98.70% higher than those to non-AI papers on average...

So, AI augmented research gathers more citations, which suggests that authors using AI in their research achieve greater impact. This is reinforced by evidence that AI augmented papers are published in higher quality journals (with Q1 journals being the highest ranked). Hao et al. report that:

...the proportion of AI papers in Q1 journals is 18.60% higher than that of non-AI papers in all journals; in Q2 journals, the AI proportion is 1.59% higher; whereas Q3 and Q4 journals hold a relatively lower proportion of papers with AI... These results indicate a heterogeneous distribution of AI-augmented papers across journals, with a higher prevalence in high-impact journals.

And AI appears to make authors more productive, as:

On average, researchers adopting AI annually publish 3.02 times more papers... and garner 4.84 times more citations... than those not adopting AI, with consistency.

All of these results seem to hold across all of the disciplines that Hao et al. consider. However, it is not all good news. Hao et al. use machine learning to create a measure of the 'breadth of scholarly attention'. Using that measure, they find that:

Compared with conventional research, AI research is associated with a 4.63% contracted median collective knowledge extent across science, which is consistent across all six disciplines... Moreover, when dividing these disciplines into more than two hundred sub-fields, the contraction of knowledge extent can be observed in more than 70% of them...

Of course, some of the differences here may be due to selection, as the types of researchers, and the types of research, involving AI use may be meaningfully different from those that don't. However, putting the selection issues aside, Hao et al. note that there is a tension between the individual researcher's incentive to produce a greater quantity of research that has higher impact, which would suggest greater use of AI, and the social incentive to produce a greater breadth of research.

So, the takeaway from this paper is that we need to consider researcher incentives, not just productivity. Specifically, this research suggests that the use of AI in research is leading to a 'prisoners' dilemma' outcome: each individual researcher acting in their own best interests (and using AI in their research) leads to an outcome that is worse for society overall (less breadth of research and more incremental gains).

Hao et al. conclude that:

The substantial academic benefits of AI use may be a driving force behind its accelerated rate of adoption; however, we also find unintended consequences from the increased prevalence of AI-augmented research. In all fields, AI-augmented research focuses on a narrower scope of scientific topics and reduces the scientific engagement of follow-on research, leading to more overlapping research work that slows the expansion of knowledge. Further, with a greater concentration of collective attention to the same AI papers, the adoption of AI seems to induce authors to converge on the same solutions to known problems rather than create new ones.

So, what is the solution here? Society probably wants research to be higher quality and have a broad scope. But individual researchers' incentives to use AI in their research appears inconsistent with that outcome. The traditional prisoners' dilemma is a repeated game (see here or here, for example), and the players of that game can avoid the worst outcome by cooperating. In this case, the researchers could cooperate by agreeing not to use AI in their research. The problem is that every researcher has an incentive to cheat on that agreement, since if they use AI, then that will be good for their career. This prisoners' dilemma is more difficult to ensure cooperation in than the traditional game, because there are not just two players who need to cooperate, but thousands (or millions). Ensuring cooperation in a prisoners' dilemma game with many players, each of whom is far better off cheating than cooperating, is almost impossible (which is why solving the problem of climate change is so difficult).

My own view is that the answer is not to keep AI out of research. That is not realistic, in the same way that it's not realistic to expect students not to use generative AI. The incentives need to be redesigned, but this will be no easy task. As long as universities, research funders, and publishers reward researchers for quantity, citations, and publication in top-ranked outlets, then we should expect more AI-augmented work, with a narrower scope than society might prefer. If we want AI to expand knowledge rather than simply accelerate competition within narrow foci, then we need institutions that also reward novelty, breadth, and the discovery of new questions. That is the economic challenge we must face up to.

[HT: Marginal Revolution]