The economics of pricing LLM tokens

Ethan Ding had a really interesting post on Substack last month, discussing his view on the future costs of tokens for large language models (LLMs), and what that means for the viability of subscription-based generative AI. I want to focus on two aspects of Ding's post. First, this (the lack of capitalisation is Ding's style):

the math has fundamentally broken.

prisoner’s dilemma for everyone else

this leaves everyone else in an impossible position.

every ai company knows usage-based pricing would save them. they also know it would kill them. while you're being responsible with $0.01/1k tokens, your vc-funded competitor offers unlimited for $20/month.

guess where users go?

classic prisoner's dilemma:

• everyone charges usage-based → sustainable industry
• everyone charges flat-rate → race to the bottom
• you charge usage, others charge flat → you die alone
• you charge flat, others charge usage → you win (then die later)

so everyone defects. everyone subsidizes power users.

Ok, so let's look at this prisoners' dilemma. Consider two AI firms (Firm A and Firm B), each with two strategies to choose from (Usage-based pricing, or flat-rate pricing). The game is outlined in the payoff table below. The payoffs are expressed in +'s and -'s, with more +'s obviously being better.

To find the Nash equilibrium in this game, we use the 'best response method'. To do this, we track: for each player, for each strategy, what is the best response of the other player. Where both players are selecting a best response, they are doing the best they can, given the choice of the other player (this is the definition of Nash equilibrium). In this game, the best responses are:

1. If Firm B chooses usage-based pricing, Firm A's best response is to choose flat-rate pricing (since ++ is a better payoff than +) [we track the best responses with ticks, and not-best-responses with crosses; Note: I'm also tracking which payoffs I am comparing with numbers corresponding to the numbers in this list];
2. If Firm B chooses flat-rate pricing, Firm A's best response is to choose flat-rate pricing (since - is a better payoff than --);
3. If Firm A chooses usage-based pricing, Firm B's best response is to choose flat-rate pricing (since ++ is a better payoff than +); and
4. If Firm A chooses flat-rate pricing, Firm B's best response is to choose flat-rate pricing (since - is a better payoff than --).

Note that Firm A's best response is always to choose flat-rate pricing. This is their dominant strategy. Likewise, Firm B's best response is always to choose flat-rate pricing, which makes it their dominant strategy as well. The single Nash equilibrium occurs where both players are playing a best response (where there are two ticks), which is where both firms choose flat-rate pricing.

Notice that both players would be unambiguously better off if they chose usage-based pricing. However, both will choose flat-rate pricing, which makes them both worse off. This is a prisoners' dilemma game (it's a dilemma because, when both players act in their own best interests, both are made worse off).

Ding notes that this is a losing proposition for all generative AI firms, and is the position that they are all in right now. They could try to cooperate and shift to usage-based pricing, but there will always be a strong incentive for the firms to cheat on any agreement and instead offer flat-rate pricing. So, any agreement will not last. Especially since there are other strategies available, which Ding goes on to discuss. The one that caught my eye was this:

use ai as a loss leader to drive consumption of aws-competitive services. you're not selling inference. you're selling everything else, and inference is just marketing spend.

the genius is that code generation naturally creates demand for hosting. every app needs somewhere to run. every database needs management. every deployment needs monitoring. let openai and anthropic race inference to zero while you own everything else.

the companies still playing flat-rate-grow-at-all-costs? dead companies walking. they just have very expensive funerals scheduled for q4.

It makes sense to play the losing prisoners' dilemma strategy, if a firm can use it to be more profitable elsewhere. Using generative AI as a loss leader, and then making more profits by selling complementary services (hosting, data management, monitoring) may be more profitable overall for the generative AI firms.

For loss leading to be successful though, two conditions need to be met. First, the loss leading service should be price elastic. That means that when price is low, many consumers are attracted to the service. That seems likely to be the case for generative AI, because when the price increases, consumers can easily switch to one of the many other generative AI platforms. Second, there must be many other complementary services for the firm to sell. The three suggestions by Ding (hosting, data management, monitoring) are all complements to generative AI (or, at least, to the ways that generative AI is being used right now). So, it seems that loss leading with generative AI may be a profitable strategy for the generative AI firms, even though it means playing out the prisoners' dilemma on pricing.

[HT: Marginal Revolution]