The economics of pricing LLM tokens

Ethan Ding had a really interesting post on Substack last month, discussing his view on the future costs of tokens for large language models (LLMs), and what that means for the viability of subscription-based generative AI. I want to focus on two aspects of Ding's post. First, this (the lack of capitalisation is Ding's style):

the math has fundamentally broken.

prisoner’s dilemma for everyone else

this leaves everyone else in an impossible position.

every ai company knows usage-based pricing would save them. they also know it would kill them. while you're being responsible with $0.01/1k tokens, your vc-funded competitor offers unlimited for $20/month.

guess where users go?

classic prisoner's dilemma:

• everyone charges usage-based → sustainable industry
• everyone charges flat-rate → race to the bottom
• you charge usage, others charge flat → you die alone
• you charge flat, others charge usage → you win (then die later)

so everyone defects. everyone subsidizes power users.

Ok, so let's look at this prisoners' dilemma. Consider two AI firms (Firm A and Firm B), each with two strategies to choose from (Usage-based pricing, or flat-rate pricing). The game is outlined in the payoff table below. The payoffs are expressed in +'s and -'s, with more +'s obviously being better.

To find the Nash equilibrium in this game, we use the 'best response method'. To do this, we track: for each player, for each strategy, what is the best response of the other player. Where both players are selecting a best response, they are doing the best they can, given the choice of the other player (this is the definition of Nash equilibrium). In this game, the best responses are:

1. If Firm B chooses usage-based pricing, Firm A's best response is to choose flat-rate pricing (since ++ is a better payoff than +) [we track the best responses with ticks, and not-best-responses with crosses; Note: I'm also tracking which payoffs I am comparing with numbers corresponding to the numbers in this list];
2. If Firm B chooses flat-rate pricing, Firm A's best response is to choose flat-rate pricing (since - is a better payoff than --);
3. If Firm A chooses usage-based pricing, Firm B's best response is to choose flat-rate pricing (since ++ is a better payoff than +); and
4. If Firm A chooses flat-rate pricing, Firm B's best response is to choose flat-rate pricing (since - is a better payoff than --).

Note that Firm A's best response is always to choose flat-rate pricing. This is their dominant strategy. Likewise, Firm B's best response is always to choose flat-rate pricing, which makes it their dominant strategy as well. The single Nash equilibrium occurs where both players are playing a best response (where there are two ticks), which is where both firms choose flat-rate pricing.

Notice that both players would be unambiguously better off if they chose usage-based pricing. However, both will choose flat-rate pricing, which makes them both worse off. This is a prisoners' dilemma game (it's a dilemma because, when both players act in their own best interests, both are made worse off).

Ding notes that this is a losing proposition for all generative AI firms, and is the position that they are all in right now. They could try to cooperate and shift to usage-based pricing, but there will always be a strong incentive for the firms to cheat on any agreement and instead offer flat-rate pricing. So, any agreement will not last. Especially since there are other strategies available, which Ding goes on to discuss. The one that caught my eye was this:

use ai as a loss leader to drive consumption of aws-competitive services. you're not selling inference. you're selling everything else, and inference is just marketing spend.

the genius is that code generation naturally creates demand for hosting. every app needs somewhere to run. every database needs management. every deployment needs monitoring. let openai and anthropic race inference to zero while you own everything else.

the companies still playing flat-rate-grow-at-all-costs? dead companies walking. they just have very expensive funerals scheduled for q4.

It makes sense to play the losing prisoners' dilemma strategy, if a firm can use it to be more profitable elsewhere. Using generative AI as a loss leader, and then making more profits by selling complementary services (hosting, data management, monitoring) may be more profitable overall for the generative AI firms.

For loss leading to be successful though, two conditions need to be met. First, the loss leading service should be price elastic. That means that when price is low, many consumers are attracted to the service. That seems likely to be the case for generative AI, because when the price increases, consumers can easily switch to one of the many other generative AI platforms. Second, there must be many other complementary services for the firm to sell. The three suggestions by Ding (hosting, data management, monitoring) are all complements to generative AI (or, at least, to the ways that generative AI is being used right now). So, it seems that loss leading with generative AI may be a profitable strategy for the generative AI firms, even though it means playing out the prisoners' dilemma on pricing.

[HT: Marginal Revolution]

Book review: Lives of the Laureates

During my travels in Europe, I managed to get through reading Lives of the Laureates, which includes autobiographical accounts written by Nobel Prize winners in economics. The source material is an ongoing lecture series hosted by Trinity University in Texas, who invite recent Nobel laureates from US institutions to reflect on their 'evolution as an economist'. The lectures are collected together in this volume.

The first edition of the book was published in 1985. I read the sixth edition, which was released in 2014. However, I note that a seventh edition was published last year. The difference between the editions is the selection of economists whose lectures are included. I was a little disappointed to find out that earlier editions included lectures by Arthur Lewis and Ronald Coase, among others. The seventh edition adds many more recent Nobel laureates, including Amartya Sen, Michael Spence, and Alvin Roth, as well as adding back in the earlier lectures that were missing from the sixth edition.

Each lecture is a standalone autobiographical account written by the laureate. However, each lecture is quite idiosyncratic, as each laureate takes a different approach to their task. Some present what is essentially an extended timeline of their research work, while others present a much more personal account. I much preferred the latter, for the additional insights into the people behind the Nobel Prizes. The lectures by Douglass North, James Heckman, and Thomas Schelling were particularly interesting to me. For instance, I learned that Schelling wrote a review of Peter Bryant's book Red Alert, which drew the attention of Stanley Kubrick and eventually became the movie Dr. Strangelove. I also got to learn a lot about the Nobel Prize winners that I didn't know much about before, including Lawrence Klein, Eric Maskin, and Peter Diamond.

There were also a number of interesting anecdotes, such as this from Franco Modigliani, about the research that introduced the now-famous Modigliani-Miller theorem:

This paper has since become quite well known, and it has been assignment to students in business and finance all over the world... this paper was never meant for students. The paper was meant to upset my colleagues in finance by arguing that the core issue that receive most attention in corporation finance, namely finding out what exactly is the optimum capital structure, was not really an issue. It didn't make any difference.

On a similar note, I hadn't appreciated just how negative the reaction to Gary Becker's work was from within economics, at least initially. For example:

The negative reaction to my work on discrimination, coupled with Frank Knight's hostility toward my article on democracy, made it clear to me that using economic analysis to discuss social and political issues was not going to be welcomed with open arms by most economists... I was surprised that the main hostility toward my work, at least as it was explicitly stated, came from economists, not non-economists.

Although the lectures are quite different, there are nevertheless some common themes, which the final chapter of the book attempts to draw together. One that stood out to me while reading was:

...the role of luck... The laureates often use the term "luck" to mean the unpredictability or unplanned path through which their career evolved. each can readily imagine some alternative turn taken early in life that would have set them on an altogether different path.

And finally:

One cannot come away from these lectures without an appreciated for the critical role of teachers, the intellectual environment, the search for rigor and relevance, and the role of happenstance in the evolution of modern economic thought.

I really enjoyed this book, and I recommend it to anyone who wants to learn a little bit more about the history of economic thought, from a variety of different perspectives to those found in traditional textbooks or more general books on the topic.

This week in research #90

This week I attended the European Regional Science Association (ERSA) conference in Athens. The ERSA conference is the largest regional science conference (surprisingly, larger than the RSAI International Conference), and this year had 930 attendees from over 50 countries. Not many would have travelled further than the small contingent of New Zealanders. However, as always, the conference was totally worth the long trip, and I've come away inspired and re-energised in my research. With such a large programme, it was impossible to attend all of the sessions that I wanted to. Nevertheless, here are some of the highlights I found from the conference:

• Nikolaos Terzidis (abstract on page 181 of the abstract book) showed that trade exposure has shifted voting patterns to the right (decreases in voting for radical left parties, and increases in voting for radical right parties) in Greece, with no apparent increase in polarisation
• Anna Lewczuk (abstract on page 540 of the abstract book) showed that exposure to the war in Ukraine (proxied by proximity to the Poland-Ukraine border) increased voting for Eurosceptic parties in Poland
• Pedro Fierro (working paper here) showed that bonding social capital (social capital within groups) was positively associated with Donald Trump's 'margin vote' in the 2020 Presidential election, while bridging social capital (social capital between groups) was negatively associated with the 'margin vote'
• In an interesting session organised by The Regional Science Academy, Nobel Prize winner Paul Krugman described the US as becoming "a hermit kingdom"
• Lukáš Makovský (abstract on page 304 of the abstract book) showed that higher house prices and construction constraints impede migration in England and Wales, and that the impacts are larger for lower-skilled households than for higher-skilled households (this research also has high relevance for New Zealand)
• Daniel Ruiz Palomo (abstract on page 411 of the abstract book) showed that Spanish municipalities that experienced a one-minute lower relative average travel time due to high-speed rail grew three percentage points more in population over a 30-year period (and I learned that Spain has the second largest high-speed rail network, behind only China)
• Jacques-François Thisse gave a wonderful and wide-ranging keynote lecture on the development of regional science and spatial economics (but was, in my view, a little unfair to economists, who hadn't totally ignored developments in regional science in the 1970s and 1980s)

Aside from the conference, here's what caught my eye in research over the past week:

• Allen and Webber (open access) test the 'wisdom of the crowd' by looking at predictions of final rank positions in the English Premier League, and find that experts and bookmakers outperformed a crowd of prediction game players
• Arrondel, Duhautois, and Laslier (with ungated earlier version here) develop a theoretical model that shows that football clubs that include not only profit but also sporting performance in their objectives end up generating more profit than others who purely maximise profit
• Hanedar, Göktan, and İnce (with ungated earlier version here) find that investors anticipated the outbreak of World War I, with bond prices following a continuously downward trend from 1911

Dame Peggy Koopman-Boyden, 1943-2025

I was really saddened to learn this week of the passing of Professor Dame Peggy Koopman-Boyden. Peggy was a long-term collaborator of mine, as I noted in this post from 2017, on the occasion of her being made a Dame Companion of the New Zealand Order of Merit.

As I noted in that 2017 post, Peggy and I may have collaborated a lot, going back to 2006, but we only published one research paper together - this 2015 working paper entitled "Labour Force Participation, Human Capital and Wellbeing among Older New Zealanders" (co-authored with Matthew Roskruge at Massey University). I won't reiterate my other comments from that 2017 post. However, in recent years, Peggy and I served together on the governance board of the Institute of Healthy Ageing, a collaboration between the University of Waikato, Health New Zealand, and other organisations.

Peggy made a number of contributions to the community as well as to academia. This Waikato Times article provides a great overview. Peggy continued to contribute widely until relatively recently, when poor health started to intervene.

I saw Peggy only a few times over the past couple of years. I'll most remember her for her ready smile, her keen intelligence and wisdom, and her frequent witty remarks. I'll miss those moments when she would ask what I had been working on, and then quip that I spent too much of my time on research that was unrelated to population ageing. It is somewhat fitting then, that the presentation I will be making at the European Regional Science Association conference tomorrow is on subnational population ageing (in Australia).

Peggy is a great loss to the New Zealand community. I am very surprised that, as yet, there are no obituaries of her online. She will be greatly missed.