Will AI make personalised pricing a reality, and is that really a bad thing?

Personalised pricing (or first-degree price discrimination) occurs when the seller sells their good or service to every consumer at a different price, as I described in this 2023 post. If executed perfectly, the seller could extract all of the consumer surplus as profits, by charging a price to every consumer that is exactly equal to the maximum the consumer is willing to pay. Fortunately for consumers, such perfect personalised pricing has remained a theoretical possibility.

But technological tools are increasingly helping firms to learn more detailed information about consumer preferences, and that allows firms to home in on consumers' maximum willingness-to-pay. The latest worry for consumers is AI, as this article in The Conversation by Patrick Dodd and Hanoku Bathula (both University of Auckland) notes:

Digital platforms can observe thousands of individual decisions. A ride-hailing platform can see which jobs a driver accepts, when they work and which incentives bring them online. A retailer can see purchases, abandoned carts and responses to discounts.

There is no strong evidence major companies already know everyone’s precise financial breaking point. But algorithmically mediated pay, personalised worker incentives, discounts and consumer offers are already real.

Notice that Dodd and Bathula also take the logic of personalised pricing for consumers, and apply it to gig-economy workers as well. Platforms such as Uber or Lyft or Doordash can increasingly use what they know about their delivery workers' preferences to determine their minimum willingness-to-accept for each delivery. The target is different (minimising how much they pay to the delivery worker, rather than maximising the price they charge the consumer), but the underlying premise of personalised pricing is the same.

Algorithms have been around for a while, though. Dodd and Bathula do not clearly lay out why they think that recent developments in AI make personalised pricing more of a reality than before. Most of what they say about 'algorithms' applies equally to statistical algorithms that have been around for years (decades, even) as to more recent developments in AI and machine learning (AI/ML). So, let me extend their argument more explicitly.

AI/ML dramatically lowers the cost of estimating individual willingness-to-pay. It can combine huge numbers of relatively weak signals about a particular consumer, learn complex patterns from the behaviour of millions of other similar consumers, experiment continually with prices and discounts, and update its estimate each time that circumstances change. That gives firms far richer models from which to estimate each particular customer's maximum willingness-to-pay (or, for their workers, to estimate their minimum willingness-to-accept). Moreover, while older statistical algorithms allowed firms to segment customers into fairly coarse categories, AI/ML allows firms to make predictions for each individual, and in real time. The better estimates from these newer models therefore allow firms to price much closer to the perfectly price-discriminating ideal. It's still not completely perfect, but it is a further improvement on what they were previously able to achieve.

Dodd and Bathula finish their article by noting the unfairness of personalised pricing. Their argument is essentially that there is asymmetry in the relationship between consumers and firms. Firms using algorithms (and now AI/ML) know increasingly more about what consumers are willing to pay, but consumers know very little about what firms are willing to accept.

However, it is worth unpacking that a bit more. Firms that don't know consumer willingness-to-pay can't raise their prices without limit, as consumers with low willingness-to-pay would stop buying from them. In practice though, personalised pricing will never be perfect. Firms may charge higher prices to consumers that they estimate have high willingness-to-pay, while offering lower prices or discounts to consumers with lower willingness-to-pay. So, relative to offering the same price to everyone, personalised pricing need not make every consumer worse off. The high-willingness-to-pay consumers are likely to be worse off, but some low-willingness-to-pay consumers may actually be better off.

Now consider which types of consumers tend to have high willingness-to-pay, and which types tend to have low willingness-to-pay. For many goods, lower-income consumers are likely, on average, to have lower willingness-to-pay, so personalised pricing could result in some of them being offered lower prices. That won't always be true though. Some lower-income consumers with few alternatives or an urgent need may have high willingness-to-pay despite having a low income. Taken together, this means that the distributional effects of personalised pricing are not necessarily straightforward. However, in some instances preventing firms from price discriminating could be making low-income consumers worse off. With that in mind, is it really fairer that firms are not allowed to offer lower prices to consumers with low willingness-to-pay?

I'm not really trying to defend price discrimination here. I'm not keen on personalised pricing for very selfish reasons - I don't want to pay more, even if I am willing to pay more! And like me, most consumers should probably not be keen on personalised pricing. But before we rail against the evils of firms price discriminating, we need to properly consider its distributional consequences. And that means thinking about which groups may be made better off by price discrimination, not just which groups are made worse off.

Read more:

• Surge pricing is coming to a supermarket near you
• What would happen if perfect personalised pricing was possible?
• Personalised pricing is now a step closer with electronic shelf labels

Can a simple email get students to study more economics?

This year, I started sending the top students in my ECONS101 class a 'student recognition letter', congratulating them on their performance and noting their overall grade and their ranking in the class. The purpose of this letter was not to try and sell these students on studying economics, but to provide them with some well-earned recognition, as well as a valuable signal for future employers that their A+ grade actually meant something (whereas in some papers, it clearly doesn't mean much).

I could have added a sentence or two encouraging those students to study more economics. They certainly have shown an aptitude for it. However, some years ago I did try to encourage the top students to study more economics, but it seemed like those efforts didn't have much impact. I stopped doing this in about 2018. And it seems I might have made a good choice, at least according to this forthcoming article in the journal Economics of Education Review (open access) by Olivia Edwards and Jonathan Meer (both Texas A&M University).

Edwards and Meer study the effect of a simple email from the professor to the top ten percent (or so) of the class in introductory economics at Texas A&M University:

...praising their performance and encouraging them to take more economics courses and considering majoring or minoring in economics if they were not already doing so.

Their analysis is based on 10,600 students who took the course between 2017 and 2023, of which 1802 received the encouragement email. Edwards and Meer apply a regression discontinuity design (RDD) approach to the analysis, essentially comparing students just above the cutoff mark for receiving the encouragement email with those just below the cutoff mark. The outcome variables Edwards and Meer look at are whether the student went on to study intermediate microeconomics, whether they majored or minored in economics, and whether they majored in an 'economics-adjacent field' (by which they mean agricultural economics or business). They also test whether the results differ by gender, under-represented minority (URM) status, and whether the student was first-in-family to study at university.

There did seem to be some effect on students going on to study intermediate microeconomics. For students around the cutoff point for receiving the encouragement email, there was:

...a relatively large and statistically significant discontinuity of 8.8 percentage points (s.e. = 3.3 percentage points), an increase of about 40 percent over the baseline level below the cutoff.

However, there was little evidence of an effect on majoring or minoring in economics, with both effects being statistically insignificant. Turning to whether there were heterogeneous effects based on demographic characteristics, the effect of the email on taking intermediate microeconomics did not differ significantly by gender, but the overall positive effect appears to have been concentrated among URM students and students who were first-in-family to attend university. The encouragement email had no effect on going onto an economics major or minor for any of the demographic groups.

Edwards and Meer describe the effect of the encouragement email as "modest". I would say it was disappointing, but not surprising. If Texas A&M is anything like Waikato, then once a student has chosen a particular major (or minor), it is actually quite difficult to induce them to switch, even if they later find that some other major or minor would be a better fit for them. In part, this is due to institutional barriers, such as the difficulty in navigating the change-of-enrolment process, or the necessity to take pre-requisites that students might have missed, or that the student may have 'spent' papers on a particular major that would be lost if they switch. It could also be that students simply don't like changing their minds (a 'status quo bias').

So, while a simple information intervention may not be enough to change students’ choice of major, it does seem capable of nudging some students into taking more economics. And on the plus side, sending an email doesn't cost the professor much, so even if it only encourages some additional enrolments in intermediate microeconomics, on a cost-benefit basis for the Economics Department, it is probably a net positive. Given these results, perhaps I was too hasty in stopping the explicit encouragement of my top students to study more economics? I may have to re-draft my student recognition letters for this trimester.

[HT: Tim Harford, last year]

Read more:

• Could closing the gender gap in economics be as simple as providing students with information?
• Disappointing results on simple information interventions to close the gender gap in economics

This week in research #142

Here's what caught my eye in research over the past week:

• Sacerdote, Staiger, and Tine (with ungated earlier version here) find that test score–optional policies harm the likelihood of admission for high-achieving applicants from disadvantaged backgrounds, meaning that the availability of test scores on an application can promote rather than hinder social mobility
• Armona et al. (with ungated earlier version here) develop and test a model of what is newsworthy to a media outlet
• Crossin et al. (open access if you set up a free account) find using data from a nationally representative sample that an estimated 71.1 percent of the New Zealand population think politicians should do more to keep people safe from alcohol harm, with majority support across the political spectrum
• Smit (open access) outlines the drawbacks to remote working that may have prevented greater internal migration from cities to the periphery in the Netherlands

Jeanna Smialek on why we should remember Maria Edgeworth

I have read several books now on the history of economic thought and, for the most part, women are conspicuously absent from those books. One obvious exception is Edith Kuiper's A Herstory of Economics (which I reviewed here), and there are a few bits in Steven Medema's The Economics Book (which I reviewed here). The latter brought both Harriet Martineau and Jane Marcet to my attention (and my two class AI tutors, Harriet in ECONS101, and Jane in ECONS102, are named after them).

A surprising omission from both books was brought to my attention by this New York Times article (ungated version here) by Jeanna Smialik: Maria Edgeworth. [*] According to Wikipedia, Maria Edgeworth was a novelist and an aunt of the much more famous 19th Century economist, Francis Ysidro Edgeworth. However, Smialek makes a strong case for the importance of Maria Edgeworth as one of the earliest voices in the developing field of economics, with her economics expressed within her fiction. However, Smialek also spends some efforts to explain why Edgeworth has been largely forgotten as an economist:

As the field professionalized, the second- and third-generation economists distanced themselves from the women and the fiction that had once helped to popularize and explain their ideas. Serious science, after all, could not possibly be for everyone. The economist Alfred Marshall referred to the women who tried to simplify economic doctrine as “parasites” in a footnote to his hugely influential textbook, “Principles of Economics,” first published in 1890 and popular throughout the 1900s...

Her literature was lost partly because of its clunkiness — its economics lessons made it harder to digest as lighter and more naturalistic stories became the style.

Her economics was lost for a different reason. Subsequent influential academics disparaged the ways that the early women of economics had presented the field. They bristled at being simplified, and they dismissed the idea that the women could have contributed something meaningful. 

Smialek also notes that Edgeworth provided editorial comments on Jane Marcet's Conversations on Political Economy, arguably the first economics textbook:

Edgeworth was as skilled an editor as she was a writer. She told Marcet that Adam Smith ought to be abridged, because he “needs it much,” whereas the utilitarian philosopher Jeremy Bentham was “absolutely impossible” to put in fewer words, because what he needed was “to be diluted.”

“The more amusing anecdote & illustration you can mix with your solid information the better,” Edgeworth wrote to Marcet. “It should be your object rather to sow seeds than to exhibit full grown plants — Your work should excite curiosity to go further.”

Clearly, there is more for us to learn about the history of economic thought from Maria Edgeworth's life and her writing. And we are in luck! Smialek has a forthcoming book about Edgeworth due to be released next month, titled The Invisible Hand of Maria Edgeworth. I'm definitely looking forward to that one, and you can expect a review of it here in due course (although, with my backlog of reading, that might not be until sometime next year!).

*****

[*] Smialek notes that Maria Edgeworth was brought to her attention by Robert Heilbroner's The Worldly Philosophers (which I reviewed here). Heilbroner's reference to Edgeworth clearly didn't make as great an impression on me, as I don't remember it!