Last month I posted about ChatGPT's impact on the labour market (based on this working paper), and concluded that:
...Zarifhonarvar's analysis is very crude and fairly speculative. We can expect some more thorough analyses, including the first studies using real-world data, to become available before long.
This literature is moving fast. Shortly after the Zarifhonarvar paper was released, this working paper by Tyna Eloundou (OpenAI) and co-authors became available. They answer a similar research question to Zarifhonarvar, identifying the jobs that are more likely to be impacted by large language models (LLMs). Specifically:
We present our results based on an exposure rubric, in which we define exposure as a measure of whether access to a GPT or GPT-powered system would reduce the time required for a human to perform a specific [Detailed Work Activity] or complete a task by at least 50 percent.
Eloundou et al. use data from the O*NET database, which:
...contains information on 1,016 occupations, including their respective Detailed Work Activities (DWAs) and tasks. A DWA is a comprehensive action that is part of completing task, such as "Study scripts to determine project requirements." A task, on the other hand, is an occupation-specific unit of work that may be associated with none, one, or multiple DWAs.
The dataset contains 19,265 tasks and 2.087 DWAs, and they use humans to code each DWA and a subset of tasks to an exposure level (no exposure; direct exposure if a LLM could reduce the time required for a DWA or task by half; or LLM+ exposure if additional software could be developed that would allow a LLM to reduce the time required by half). In a particularly fitting research method, they then use GPT-4 to code all of the DWAs and tasks. The human and GPT-4 ratings are quite similar (for those DWAs and tasks that were coded by both).
Then, Eloundou et al. classify jobs in terms of their exposure to ChatGPT, based on the DWAs and tasks associated with each job. They find that:
...approximately 19% of jobs have at least 50% of their tasks exposed when considering both current model capabilities and anticipated tools built upon them... Accounting for other generative models and complementary technologies, our human estimates indicate that up to 49% of workers could have half or more of their tasks exposed to LLMs.
But which jobs are most exposed? Eloundou et al. note that:
Occupations with higher wages generally present with high exposure, a result contrary to similar evaluations of overall machine learning exposure... When regressing exposure measures on skillsets using O*NET’s skill rubric, we discover that roles heavily reliant on science and critical thinking skills show a negative correlation with exposure, while programming and writing skills are positively associated with LLM exposure...
We analyze exposure by industry and discover that information processing industries... exhibit high exposure, while manufacturing, agriculture, and mining demonstrate lower exposure.
There is nothing too surprising in these results, and they accord well with the earlier work by Zarifhonarvar, albeit using research methods that are more credible. Eloundou et al. conclude that "GPTs are GPTs", meaning that GPTs are a general purpose technology, in that they:
...meet three core criteria: improvement over time, pervasiveness throughout the economy, and the ability to spawn complementary innovations...
The first criterion is clearly met, as will be obvious to anyone who has followed this topic over the last six months. Eloundou et al.'s paper provides evidence for the latter two criteria.
What implications does this have? It would be attractive to infer that workers in high-wage, high-skill (particularly programming and writing skills) are at risk. However, as my students noted in a recent Economics Discussion Group meeting, it is not at all clear yet whether LLMs will be a labour-replacing technology, or a labour-augmenting technology. Will LLMs replace human labour, leading to fewer of the jobs that are most exposed to them? Or will LLMs augment those jobs, making the workers radically more productive and efficient, and opening new job opportunities? In order to understand that, we need to look at real-world cases of job change resulting from LLMs. More on that in my next post (Update: see here].
Read more:
No comments:
Post a Comment