In an article in The Conversation earlier this week, Nicholas Davis (University of Technology Sydney) wrote about information asymmetry as it relates to large language models (like ChatGPT):
A lack of skills and experience among decision-makers is undoubtedly part of the problem. But the rapid pace of innovation in AI is supercharging another challenge: information asymmetry.
Information asymmetry is a simple, Nobel prize-winning economic concept with serious implications for everyone. And it’s a particularly pernicious challenge when it comes to AI...
AI creates information asymmetries in spades. AI models are technical and complex, they are often embedded and hidden inside other systems, and they are increasingly being used to make important choices.
Balancing out these asymmetries should deeply concern all of us. Boards, executives and shareholders want AI investments to pay off. Consumers want systems that work in their interests. And we all want to enjoy the benefits of economic expansion while avoiding the very real harms AI systems can inflict if they fail, or if they are used maliciously or deployed inappropriately.
Davis is generally talking about information in relation to AI safety, but I want to take this post in a slightly different direction first, about AI quality. I will return to AI safety at the end.
When we consider AI quality, information asymmetry will cause a problem of adverse selection, ultimately causing the market for high-quality large language models to fail. So, let's consider how this market fails. There is information asymmetry in relation to the quality of large language models. AI firms know whether their large language model (LLM) is high quality or not, but users do not know. The 'quality' of a large language model is private information. Because of this information asymmetry, a LLM user should assume that any LLM is low quality. This creates a pooling equilibrium, where the user is only willing to pay however much they value access to a low-quality LLM. AI firms with high-quality LLMs, which are more expensive to train and run, will not be willing to accept users who are only willing to pay a small amount. These AI firms with high-quality LLMs drop out of the market. The only AI firms that are left will be those with low-quality LLMs. The market for high-quality LLMs fails.
A key part of the problem here is that LLMs may be experience goods. Experience goods are goods where the consumer doesn't know the quality (or some other important attribute) until after they have consumed it (they are called experience goods, because consumers don't find out what they have really bought until they have experienced it). The quality of a LLM isn't revealed to the user until after they have used it, and seen what the LLM outputs in response to a given prompt. However, users are not experts, and they might not even know the quality of the output after they receive it. That would make LLMs not experience goods, but credence goods (goods where some of the characteristics, the credence characteristics, are not known even after the good has been consumed). It seems likely that for most non-expert LLM users, LLMs will be a credence good.
So, how can the adverse selection problem be solved for LLMs? In other words, how can we move from the pooling equilibrium to a separating equilibrium (where LLM users can separate the high-quality LLMs from the low-quality LLMs)? There are two options: (1) screening; and (2) signalling. Screening involves the uninformed party (the LLM user) trying to reveal the private information (the quality of the LLM). This is unlikely to work well, because the LLM user typically isn't an expert. The only way that they can reveal the quality of the LLM themselves is to try it out, but if LLMs are actually a credence good, then even trying them out won't reveal their quality.
That leaves signalling, which involves the informed party (the AI firm) credibly revealing the private information. Only an AI firm with a high-quality LLM would want to attempt signalling, to try and reveal that their LLM is high quality (and AI firm with a low-quality LLM is unlikely to want to reveal that their LLM is low quality). In order for a signal to be effective, it needs to meet two conditions. First, it must be costly. And second, it must be costly in such a way that the AI firms with low-quality LLMs wouldn't want to attempt the signal.
What might be good signals for an AI firm with a high-quality LLM? Firms that sell experience goods have come up with a number of options, including free trials, testimonials from other customers, branding or advertising. In this context, a free trial might not work. If LLMs are a credence good, then a free trial won't reveal the private information (and we would expect to see AI firms with low-quality LLMs also offering free trials). So, while free trials are costly (they come with an opportunity cost of foregone income for the AI firm), they don't appear to be costly in a way that AI firms with low-quality LLMs wouldn't want to attempt them.
Testimonials from other customers might be an option, but only if the LLM user can trust those other customers. So, testimonials from acknowledged experts in AI might be worthwhile, but testimonials from other non-expert users might not. Are testimonials costly to the firm? Probably not greatly, but they will be more costly for an AI firm with a low-quality LLM. They might have to pay for a testimonial, whereas an AI firm with a high-quality LLM might receive praise from an expert without payment. The problem here, though, is that a non-expert user may not be able to tell the 'experts' giving testimonials apart, and so may not trust any testimonial.
What about branding or advertising? These are costly to a firm, but it's unlikely that they are costly in a way that would prevent AI firms with low-quality LLMs from attempting them.
All of this is bad news for the LLM user. They can't tell high-quality LLMs and low-quality LLMs apart, they probably can't use screening to help tell them apart, and the AI firms with high-quality LLMs probably can't use signalling to reveal that their LLM is high quality. We're left in a situation where the LLM user just has to assume that any given LLM is low quality (the pooling equilibrium).
So, what's left? Davis suggests that:
Well-designed guardrails will improve technology and make us all better off. On this front, the government should accelerate law reform efforts to clarify existing rules and improve both transparency and accountability in the market.
Of course, this is intended as a solution to information asymmetry about AI safety, not a solution to the adverse selection problem related to AI quality. However, now that we've considered the market failing in relation to AI quality because quality is a credence characteristic, it might be worth considering: is AI safety also a credence characteristic?
If AI safety is a credence characteristic, then safety isn't revealed even after the LLM has been used. Then we end up in a similar situation for AI safety as we do for AI quality, in that there is little that users can do to separate the safe LLMs from the unsafe LLMs. We would have a AI safety pooling equilibrium, where all LLM users would have to treat LLMs as potentially unsafe. Since LLM users would not be willing to pay more for LLMs that are safe (since they couldn't be sure about safety), and because safety presumably costs AI firms more, the AI firms with safe LLMs will start to drop out of the market. That would leave only the unsafe (but regulated) LLMs available, which seems like a suboptimal outcome. This isn't an argument against AI safety regulation, but hopefully provides some food for thought about its possible consequences.
No comments:
Post a Comment