This week in research #135

Here's what caught my eye in research over the past week:

• Katz and Jung (open access) estimate the macroeconomic impact of generative AI using cross-country data over the period 2022 to 2025, finding that generative AI contributed to increasing the productivity of most workers, regardless of their education, contract type, full or partial work time, and vulnerability level
• Dills and Raghav (with ungated earlier version here) find that state legalization of recreational marijuana in the US substantially reduced arrests and disciplinary incidents for drug law violations on college and university campuses
• Wang and Wong find an overall negative association between financial literacy and use of Buy-Now-Pay-Later services in the US
• Funahashi and Cardazzi (with ungated earlier version here) find that sports stadiums increase the value of nearby properties in Japan
• Sayre (with ungated earlier version here) finds no evidence that Airbnb entry leads to a substantial increase in eviction filings overall, but that there is a modest increase in eviction filings in neighbourhoods with a high poverty rate and/or a high concentration of renter households
• Hampole, Truffa, and Wong (with ungated earlier version here) find that female MBAs are 24 percent less likely than male MBAs to enter senior management within 15 years of graduation, and that having a larger proportion of female MBA section peers increases the likelihood of entering senior management for women but not for men

Could prosecuting STI transmission increase infections?

In my ECONS102 class this week, we covered unintended consequences - where an incentive is created that works against what was originally intended. One of my favourite examples is the familiar (but possibly apocryphal) story about cobras in Delhi, as I noted in this 2015 post:

The government was concerned about the number of snakes running wild (er... slithering wild) in the streets of Delhi. So, they struck on a plan to rid the city of snakes. By paying a bounty for every cobra killed, the ordinary people would kill the cobras and the rampant snakes would be less of a problem. And so it proved. Except, some enterprising locals realised that it was pretty dangerous to catch and kill wild cobras, and a lot safer and more profitable to simply breed their own cobras and kill their more docile ones to claim the bounty. Naturally, the government eventually became aware of this practice, and stopped paying the bounty. The local cobra breeders, now without a reason to keep their cobras, released them. Which made the problem of wild cobras even worse.

Just because the consequences of a policy are unintended, that doesn't necessarily mean that they are unforeseen. Sometimes, we can anticipate what will go wrong with a particular policy. And it's not just policies that can go wrong. Any change in costs or benefits that alters people’s incentives can produce unintended consequences. As an example, consider this recent article in The Conversation by Bridget Haire and David Carter (both University of New South Wales):

In an Australian first, a Canberra man has been convicted for giving genital herpes to a sexual partner...

This recent case represents a significant expansion of criminal law into sexual health. It sets an unhelpful legal precedent, and undermines successful public health messages.

Decades of research have concluded that prosecuting disease transmission doesn’t reduce infection and may make things worse...

But criminalising transmission can create perverse incentives not to seek medical care and treatment. If a person genuinely doesn’t know their status, it can be more difficult to prove “reckless” transmission.

The intuitive case for punishment is especially strong in this case: the man knew his status, denied having an STI when directly asked, and repeatedly had unprotected sex with his partner. However, the punishment itself will change incentives for other people.

Ideally, we want people to know their STI status. For curable STIs, diagnosis enables treatment. For example, for infections such as herpes, it allows people to use medication and other precautions that reduce the risk of further transmission.

At one level, it makes sense to punish people who knowingly infect others with an STI. That creates a strong disincentive to transmit STIs to other people. However, criminalising STI transmission also reduces the incentive to get tested, because a person not knowing that they are infected might be able to use their lack of knowledge of their infection status as a defence in a criminal case. So, we might expect that fewer people would get tested for STIs. So, on the one hand there are disincentives to transmit STIs, but on the other hand there are disincentives to find out whether you are infected with an STI, which leads to move STI transmission. If the latter effect is larger, then overall there could be higher prevalence of STIs and greater incidence of new infections.

And so, rather than reducing STI infections, criminalising those who transmit STIs may have the unintended consequence of increasing STI infections overall.

Why rising honey prices may increase kiwifruit orchard costs

This week, my ECONS102 class covered rational behaviour, one aspect of which is the cost-benefit principle: that when evaluating mutually exclusive alternatives, a rational decision-maker will choose the alternative that offers the greatest net benefit (the greatest difference between benefits and costs). So, it was interesting to see a good example of this in The New Zealand Herald just last month:

There’s growing competition for beehives as honey prices sweeten again and kiwifruit orchards continue to grow...

[Beekeeper Liam Gavin] said renewed confidence in honey production is seeing some pivot away from pollination.

“I sort of describe it as the tug of war between honey and pollination.

“Both are needing more beehives. So which one, where are they going to go? And that’ll all be down to, like, region-specific [stuff], and what people like to do in terms of how they beekeep.”...

With honey prices coming back up, [New Zealand Kiwifruit Growers Incorporated chief executive Colin] Bond expected more beekeepers would prioritise honey over pollination, which would create a challenge for kiwifruit growers.

Beekeepers can position their hives primarily to generate income from honey production, or primarily to generate income by providing pollination services. Thus, for a particular hive at a particular time, honey production and paid pollination are mutually exclusive alternatives.

A rational beekeeper, applying the cost-benefit principle, would compare the expected net benefit from using their hives for pollination with the expected net benefit from using them for honey production. That comparison would include pollination fees, expected honey revenue, transport and feeding costs, risks to hive health, and other relevant costs and benefits. As honey prices increase, the opportunity cost of committing hives to pollination increases. Ceteris paribus (holding all else constant), as honey prices increase fewer hives will be offered for pollination.

So, if kiwifruit growers (and other farm and orchard businesses that depend on pollination) want to secure enough hives for pollination, they will probably need to offer higher pollination fees. That would raise their pollination costs and, consequently, their overall orchard operating costs.

Generative AI and grade inflation

If generative AI can produce assessed work that earns higher marks than a student would earn by themselves, widespread use of generative AI should increase measured grades even in the absence of equivalent learning gains. In other words, we might expect grade inflation to occur as a result of students using generative AI, and that grade inflation to not reflect improved learning. To what extent is this generative AI-induced grade inflation occurring? That is the question addressed by this recent working paper by Igor Chirikov (UC Berkeley). 

Specifically, Chirikov looks at the change in the grade distribution across 319 courses (and over 500,000 enrolments) at "a large, selective public research university in Texas", covering the period from 2018 to 2025. He follows the labour economics literature by measuring 'task exposure' to AI, using the share of required tasks in each course's Fall 2022 syllabus that involved writing or coding - areas in which generative AI is particularly capable and could be used as a substitute for students' own efforts. Using a difference-in-differences (DID) approach, Chirikov compares the difference in the share of A grades and GPA between years before and including 2022 (when ChatGPT was released) and more recent years up to 2025, between courses more or less exposed to generative AI. He finds that:

...grades rose substantially in high-exposure courses after 2022: the share of A grades increased by 13 percentage points (about 30% relative to the 2022 baseline) and GPA by 0.12 points, accompanied by compression of the grade distribution.

Looking at the shares of other grades, Chirikov finds that:

The share of A- grades fell by 4 percentage points, the share of B+ grades by 3 percentage points, and the shares of B and below show smaller and mostly insignificant changes.

So, courses that were more exposed to generative AI have seen a greater increase in the share of A grades and a greater decrease in the share of lower grades than courses less exposed to generative AI. Chirikov then turns to exploring the mechanism that underlies the change, by extending the DID approach to also compare courses that place greater or lesser assessment weight on homework tasks (in what is called a 'triple differences' approach). In that analysis, he finds that:

...above-median homework courses show an additional 16 percentage point increase in the share of A grades relative to below-median courses with the same level of AI exposure. The effect on GPA follows a similar pattern, with an additional 0.13 point increase in high-homework courses, though this estimate is less precisely estimated...

In above-median homework courses, the share of other grades declines significantly with AI exposure relative to below-median homework courses.

So, the observed pattern of grade inflation, with grades near the top of the distribution shifting towards As to a greater extent in courses that have greater homework weight, provides strong evidence consistent with generative AI contributing to the observed grade inflation, principally by substituting for student effort on unsupervised assessment tasks.

We should be cautious about over-interpreting the results from this study. It is based on results from a single university, and it measures course-level task exposure to generative AI, rather than students' actual use of generative AI. Nevertheless, the results are consistent with what we would expect, and are likely to hold in other contexts.

Given that, how should universities adapt to solve this issue? The obvious response is to move to more invigilated, in-person assessment. However, Chirikov cautions that:

Not all skills can be meaningfully evaluated under exam conditions: the ability to produce a well-researched essay, develop a software project, or conduct an empirical analysis requires sustained engagement that timed in-person assessments cannot capture. Restricting assessment to formats that are AI-proof risks measuring a narrower set of capabilities than the ones courses are designed to develop, potentially undermining the learning goals that graded work is meant to serve. A more promising direction is to redesign assessments so that AI use is either structurally constrained by the task or purposefully incorporated into it, for example, by requiring students to document their process, justify their choices, or demonstrate understanding through follow-up interaction.

In my view, the optimal response for universities is a combination of invigilated in-person assessment and authentic assessments in which AI use is permitted (or even encouraged) and evaluated. The appropriate balance depends on the specific learning outcomes for each course. However, as I have argued before, one approach is to scaffold students through their studies, with lower-level courses that rely more on developing core knowledge, relying less on generative AI and therefore using more secure assessment, while higher-level courses increasingly incorporate generative AI use explicitly into the assessment. This also requires scaffolding students through learning how best to apply generative AI at each level.

It almost goes without saying that we cannot simply continue to assess students as we always have done. Generative AI has broken key elements of the assessment toolkit that we previously used. This is not a new observation. However, evidence that generative AI may be accelerating grade inflation makes it even more imperative that assessment practices are updated to better reflect the availability of generative AI.

Read more:

• Teaching evaluations and grade inflation
• Grade inflation is harming students' learning
• A few papers on grade inflation at universities
• Grade inflation and college completion in the U.S.
• More on teaching evaluations and grade inflation
• Grade inflation at New Zealand universities, and what can be done about it
• Who is morally responsible for grade inflation?