The economic impacts of AI (referencing my previous post) are driven by who uses AI tools, and how they use them. We got an sense of this from this working paper by Handa et al., which used data from Claude. However, by far the most widely used generative AI tool is ChatGPT, so I read this new NBER working paper by Aaron Chatterji (Duke University) and co-authors with a lot of interest (see also this blog post by David Deming, one of the coauthors). Many of the co-authors are with OpenAI, which gave them privileged access to user data from ChatGPT. Having said that though, the authors are very clear and very detailed in pointing out the steps they took to ensure data privacy was maintained (and in this matter, this paper is a model for others to follow).
Their main data is made up of:
...a random selection of messages sent to ChatGPT on consumer plans (Free, Plus, Pro) between May 2024 and June 2025.5 Messages from the user to chatbot are classified automatically using a number of different taxonomies: whether the message is used for paid work, the topic of conversation, and the type of interaction (asking, doing, or expressing), and the O*NET task the user is performing.
Using this dataset (and some related datasets, including one that matches users to the demographic details, while maintaining confidentiality), Chatterji et al. document a lot of important descriptive facts about ChatGPT users (on consumer plans), as well as trends over time.
First, they document the by-now well-known exponential growth of ChatGPT use over time, summarised in Figure 3 from the paper:
Not only has ChatGPT use grown over time, but it has also grown within every cohort of users over time (where cohorts are defined by how long ago users first started using ChatGPT). The early adopters still use ChatGPT the most, but every subsequent cohort has increased use over time, Looking at Figure 5 in the paper, it looks to me like there is a noticeable spike in about March-April 2025, when the o3 model was released:
Turning to the use of ChatGPT for work, Chatterji et al. report that:
...both types of queries grew rapidly between June 2024 and June 2025, however non-work-related messages grew faster: 53% of messages were not related to work in June 2024, which climbed to 73% by June 2025.
Interestingly, later cohorts of users have a greater share of non-work messages than earlier cohorts. However, the differences between cohorts are relatively small, and a majority of ChatGPT messages are non-work-related for every cohort. Nevertheless, there is a lot of ChatGPT-related working going on!
What sort of work? Chatterji et al. next look at the topics of conversations with ChatGPT, finding that for work-related messages:
About 40% of all work-related messages in July 2025 are Writing, by far the most common Conversation Topic. Practical Guidance is the second most common use case at 24%. Technical Help has declined from 18% of all work-related messages in July 2024 to just over 10% in July 2025.
'Writing' includes things like editing or summarising text, or translating. 'Practical Guidance' includes things like how-to advice, and tutoring or teaching. 'Technical Help' includes things like calculations, programming, or data analysis. Including non-work-related conversations:
The three most common Conversation Topics are Practical Guidance, Seeking Information, and Writing, collectively accounting for about 77% of all ChatGPT conversations.
'Seeking Information' is basically using ChatGPT as a replacement for web search (a use case that I have become particularly fond of ever since ChatGPT started routinely providing web links in its responses). Of interest to educators should be this:
Education is a major use case for ChatGPT. 10.2% of all user messages and 36% of Practical Guidance messages are requests for Tutoring or Teaching.
Of course, that won't count the ChatGPT conversations that relate to the completion of assessments, which are more likely to fall into the 'Writing' or 'Technical Help' categories.
Chatterji et al. then look at user intent, based on a categorisation of messages into 'asking' (seeking information from ChatGPT), 'doing' (asking ChatGPT to complete a task), and 'expressing' (anything else). For work-related messages, they find that:
Doing constitutes nearly 56% of work-related queries, compared to 35% for Asking and 9% for Expressing.
That contrasts with what they see when looking at all messages, where 49% of messages were 'asking' and 40% were 'doing'. Interestingly, 'doing' messages are declining as a share over time, while 'expressing' are increasing. I would have thought that 'asking' messages would have increased, but there is only slight evidence for that (obviously I am extrapolating from my own experience!).
The work activities results are quite detailed, so I won't discuss them in detail here. However, Chatterji et al. provide the following summary:
We find that about 81% of work-related messages are associated with two broad work activities: 1) obtaining, documenting, and interpreting information; and 2) making decisions, giving advice, solving problems, and thinking creatively.
Turning to the demographics of ChatGPT users and their use of ChatGPT, Chatterji et al. report that:
...a significant share (around 80%) of the weekly active users (WAU) in the first few months after ChatGPT was released were by users with typically masculine first names. However, in the first half of 2025, we see the share of active users with typically feminine and typically masculine names reach near-parity. By June 2025 we observe active users are more likely to have typically feminine names. This suggests that gender gaps in ChatGPT usage have closed substantially over time.
We also study differences in usage topics. Users with typically female first names are relatively more likely to send messages related to Writing and Practical Guidance. By contrast, users with typically male first names are more likely to use ChatGPT for Technical Help, Seeking Out Information, and Multimedia (e.g., modifying or creating images).
Looking at differences by age group:
Among those who self-report their age, around 46% of the messages in our dataset are accounted for by users 18-25.
A higher share of messages are work-related for older users. Work-related messages comprised approximately 23% of messages for users under age 26, with this share increasing with age.
Perhaps younger users are more likely to disclose their age? Having said that, I don't think anyone would be surprised by those results. Nor would they be surprised by the results by level of education:
Educated users are much more likely to use ChatGPT for work. 37% of messages are work-related for users with less than a bachelor’s degree, compared to 46% for users with exactly a bachelor’s degree and 48% for those with some graduate education. Those differences are cut roughly in half after adjusting for other characteristics, but they are still statistically significant at the less than 1 percent level. Educated users are more likely to send work-related messages.
The results by occupation are more interesting. Chatterji et al. report that:
...the unadjusted work shares are 57% for computer-related occupations; 50% for management and business; 48% for engineering and science; 44% for other professional occupations; and only 40% for all non-professional occupations. Regression adjustment moves these figures around slightly, but the gaps by occupation remain highly statistically significant. Users in highly-paid professional occupations are more likely to send work-related messages.
The 'regression adjustment' refers to using the results from a multiple regression model that controls for age, gender, education, and some other variables. Looking at user intent and conversation topics by occupation, Chatterji et al. find that:
...users in highly paid professional occupations are more likely to use ChatGPT for Asking rather than Doing... This is especially true in scientific and technical occupations. 47% of the work-related messages sent by users employed in computer-related occupations are Asking messages, compared to only 32% for non-professional occupations. These differences shrink somewhat with regression adjustment, but remain highly statistically significant...
Writing is especially common for users employed in management and business occupations, accounting for 52% of all work-related messages. Writing is also relatively common in non-professional and other professional occupations like education and health care, accounting for 50% and 49% of work-related messages respectively. Technical Help constitutes 37% of all work-related messages for users employed in computer-related occupations, compared to 16% in engineering and science and only about 8% for all other categories.
Chatterji et al. note that:
Across all occupations, ChatGPT usage is broadly focused on seeking information and assistance with decision-making.
People use ChatGPT for what it is best suited for. So, what does this all mean for the economic impact of AI (and specifically, the economic impact of ChatGPT)? Chatterji et al. conclude that:
...our findings suggest that ChatGPT has a broad-based impact on the global economy. The fact that non-work usage is increasing faster suggests that the welfare gains from generative AI usage could be substantial... Within work usage, we find that users currently appear to derive value from using ChatGPT as an advisor or research assistant, not just a technology that performs job tasks directly. Still, ChatGPT likely improves worker output by providing decision support, which is especially important in knowledge-intensive jobs where productivity is increasing in the quality of decision-making.
None of that answers the stream of questions posed by Kevin Bryan (which I outlined in my previous post). Nevertheless, it is important to recognise both how widely used ChatGPT is (in case you've been living under a rock for the last three years), and how it is used, particularly by workers in their daily tasks. This research provides us with broad answers, which can now be supplemented with more detailed analyses of particular industries and occupations.
[HT: Marginal Revolution]
No comments:
Post a Comment