In a recent Wall Street Journal article (ungated version here), Avinash Collis (Carnegie Mellon University) and Erik Brynjolfsson (Stanford University) report that:
In late 2024, a nationally representative survey of U.S. adults revealed that 40% were regular users of generative AI. Our own survey found that their average valuation to forgo these tools for one month is $98. Multiply that by 82 million users and 12 months, and the $97 billion surplus surfaces.
This estimate of $97 billion per year provides a measure of the economic impact of AI. That is about 0.33 percent of US GDP (which was about $29.3 trillion in 2024). That is somewhat less that I would have expected. However, to put that $97 billion in context, it is nearly the same size as the 'value added' (the industry-level equivalent of GDP) of the entire motion picture and sound recording industry (which was $119 billion in 2024).
Comparisons such as that may not be sensible though, because we aren't quite comparing apples with apples. GDP is really a measure of production, whereas the value generated by ChatGPT estimated by Collis and Brynjolfsson is closer to a measure of consumer surplus. ChatGPT does contribute to GDP already of course, because each user who has a paid account pays for access. However, what they pay is far less than the value they receive from access to ChatGPT. And some consumers are paying nothing at all.
This problem, and a potential solution, are outlined in this new article by Brynjolfsson, Collis, and co-authors, published in the American Economic Journal: Macroeconomics (ungated earlier version here). They explain the problem as follows:
New, sometimes very specialized, goods appear with increasing rapidity... and digital goods (such as information and entertainment services) are increasingly available at zero price, reflecting their very low marginal costs of replication and distribution... the positive quantities of these goods that are consumed have a measured price of zero and measured value of zero in the conventional national accounts even if they create considerable consumption value for consumers. A related difficulty arises in the valuation of new goods, since there is no observed price for the period before their appearance. Despite the increasing relevance of new and free goods, the value to consumers is not reflected in standard statistical agency reports for GDP or derivative metrics like productivity, which are typically defined in terms of GDP.
Brynjolfsson et al. provide a framework for estimating the welfare contribution from new zero-priced goods, as well as the quality improvement of existing goods. The framework itself is quite mathematical and not for the faint-hearted. However, the basic premise is that the welfare that is generated by a good that is offered for free can be estimated by estimating how much consumers would be willing to accept to give up access to that good for a period of time (notice that this is what Collis and Brynjolfsson talk about in their WSJ article in relation to generative AI).
There are various ways that can be used to estimate what consumers would be willing to accept to give up a free service. However, the challenge is that the estimates need to be credible. Brynjolfsson et al. provide two examples of incentive-compatible experiments. In these experiments, the research participants might really have to give up the service they were being asked to value, which provides a strong incentive for them to provide their 'real' valuation of what they would accept. The first experiment was conducted online and used to value Facebook (this is research that I blogged about back in 2018). In this case:
In the experiment, each participant was asked to make a single discrete choice between two options: (i) keep access to Facebook or (ii) give up Facebook for one month and get paid $E. We allocated participants randomly to 1 of 12 price points: E ∈ {1, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 1,000} —that is, we observe at least 200 participants per price point. Before participants made the decision, we informed them that their decisions were consequential such that we would randomly pick 1 out of every 200 participants and fulfill that person’s selection. We also informed them about how we can monitor their Facebook online status remotely. In order to check if the selected participants gave up Facebook and qualified for the payment, we monitored their online status on Facebook for 30 days...
Research participants were only paid if they didn't log onto Facebook for the 30 days. In their second example, Brynjolfsson et al. conducted a lab experiment with students in the Netherlands. In this case:
We asked participants to state the minimum amount of money they would request in order to give up their smartphone camera (both main camera and front camera) for one month. Participants were informed that this amount would serve as a bid in a lottery. If their minimum bid to forgo their camera would be higher than a random price, drawn from a uniform distribution, they could keep access to their smartphone camera but would not receive any cash...
In order to induce incentive compatibility and make the answers consequential, we provided further information that 1 out of 50 participants would be selected for the lottery and that we would block their smartphone cameras with a special sealing tape if their bid was successful... The sealing tape would break if the participants tried to peel it off so that it was not possible to reapply it. We also signed the tape so that it was not possible to buy the same type of seal and reapply a seal. If, after the one-month period, the original seal was still intact participants were rewarded with the money and the seal could be removed.
Notice that in both cases, the research participants might really have to give up what they are being asked to value. Collis and Brynjolfsson no doubt did something similar to derive their estimate of the value of generative AI.
Anyway, coming back to the Brynjolfsson et al. paper, their purpose is to create a new measure, which they term GDP-B, that includes the benefits of goods and services that have high marginal value to consumers, but are offered at low or no cost to them. This is not limited to the particular examples that Brynjolfsson et al. cite (which include not just Facebook and smartphone cameras, but also Instagram, Snapchat, Skype, WhatsApp, digital maps, LinkedIn, and Twitter). Even that list is very incomplete of course.
And, given that every good or service generates consumer surplus, the approach could be extended to every good or service. No doubt that would be the ideal, so that GDP-B captures the consumption benefits of all goods and services. However, the data collection task required to estimate consumer value for every good and service would be enormous, dwarfing the efforts already taken to measure the Consumer Price Index (which doesn't survey the price of every good and service).
Adopting an approach that only accounts for some goods and services seems to me to be suboptimal. And that is what makes the comparisons a bit problematic. Brynjolfsson et al. compare traditionally-measured GDP growth with growth based on their measure GDP-B, which includes the benefits from Facebook (or smartphone cameras). However, all they can really show there is that growth is higher when the benefits from Facebook (or smartphone cameras) are included. I would take the question of how much higher growth is with a grain of salt - they aren't accounting for the benefits of all of the other goods and services that they haven't valued in the same way (for some of which, the benefits might have declined, because consumers value them less, leading to lower growth).
None of this is to say that we shouldn't be finding better ways to measure welfare than GDP, which has long been acknowledged as a very incomplete measure. Brynjolfsson et al. have provided a measure that goes beyond GDP, which is what many have been calling for, for some time.
[HT: Marginal Revolution, here for the WSJ article, and here for the AEJ:Macro paper]
Read more:
No comments:
Post a Comment