Sunday, 7 March 2021

Performance-based scholarships and incentives

There are fundamentally two different types of scholarships that can be awarded to university students. First, scholarships can be awarded with no strings attached. Essentially these are grants that the students can use for any purpose (or sometimes they are linked to tuition or accommodation), but there is no need for students to do anything further to retain their scholarship. Second, scholarships can be performance-based, in which case students must attain a minimum grade point average (or similar) in order to remain eligible and continue to receive their scholarship. The idea behind a performance-based scholarship is that it increases the incentives for students to maintain a high level of performance, compared with a scholarship with no performance requirement.

The theory underlying a performance-based scholarship is simple. It relies on the assertion that people respond to incentives. When economists say that, they mean that when the costs of doing something increase, we tend to do less of it. And if the costs of doing that thing decrease, we tend to do more of it. The reverse is true of benefits - when the benefits doing something increase, we tend to do more of it, and when the benefits decrease, we tend to do less of it. In the case of the performance-based scholarship, the benefits of studying hard are increased, because the reward for studying is that the student has a higher probability of retaining their scholarship. [*]

Does it work in practice? In a 2018 article published in the journal Education Finance and Policy (ungated earlier version here), Lisa Barrow (Federal Reserve Bank of Chicago) and Cecilia Rouse (Princeton University) provide some evidence that it does. They used data on the California Scholarship Program. As they describe it:

High school seniors in California were randomly assigned to treatment and control groups where the treatments (the incentive payments) varied in length and magnitude and were tied to meeting performance, enrollment, and/or attendance benchmarks...

The incentive varied in length (as short as one semester and as long as four semesters), size of scholarship (as little as $1,000 and as much as $4,000), and whether there was a performance requirement attached to it.

The randomisation allows Barrow and Rouse to compare students who did receive the scholarship with those who didn't. The randomisation across performance-based scholarship (PBS) and not performance-based scholarship, and different durations and values, allows Barrow and Rouse to test whether a performance-based scholarship increased the incentive to study hard, and whether higher scholarship amounts increase the incentive further. They have data on about 6600 students (who were seniors in high school at the time of randomisation), across two cohorts (2009 and 2010). They find that:

PBS-eligible students were 5.2 percentage points more likely than the control group to report ever enrolling at a postsecondary institution, a difference that is statistically significant at the 1 percent level. Further, the PBS-eligible students reported studying about eight minutes more per day than those in the control group, were 7.3 percentage points more likely to have been prepared for class in the last seven days, and were 6.7 percentage points more likely to report attending all or most of their classes in the last seven days.

All of those results seem to suggest an increased incentive to study hard, as PBS recipients were spending more time studying than students who received no scholarship at all. However, when comparing PBS recipients and non-PBS scholarship recipients:

We generally find the impacts are larger for those eligible for a PBS than for those offered a non-PBS, however, in most cases we are unable to detect a statistically significant difference.

In other words, the incentives appear to be the same for performance-based and non-performance-based scholarships. The question then becomes, do PBS recipients study smarter, rather than harder? It appears that might be the case:

...PBS eligibility may induce participants to concentrate more on their studies by encouraging them to employ more effective study strategies, making the time devoted to educational activities more productive. Similarly, by raising their academic self-efficacy the scholarships may also induce students to be more engaged with their studies... We estimate that eligibility for a PBS had positive and statistically significant impacts on these dimensions that range from 12 to 22 percent of a standard deviation. Note as well that the impacts on learning strategies and academic self-efficacy for those selected for a non-PBS were substantially smaller than those selected for a PBS, consistent with increased academic effort on the part of PBS-eligible individuals.

So, the PBS recipients were using more effective study strategies than non-PBS recipients, even if they weren't studying harder. Coming back to the comparison with students who didn't receive any scholarship, the extra time spent studying arises because:

...participants accommodated increased time spent on educational activities by spending (statistically) significantly less time on leisure activities, including reducing the number of nights out for fun during the past week.

Moving on to the size of the scholarship, the results are much less clear-cut:

Interestingly, we do not find large differences in the effect of PBS eligibility related to the size of the scholarship. Students who were eligible for a $500 per semester scholarship responded similarly on most outcomes to students who were eligible for a $1,000 per semester scholarship, suggesting that larger incentive payment amounts did not lead to larger impacts on student effort.

This may seem a little surprising. However, Barrow and Rouse suggest some explanations for the lack of increased incentive associated with larger scholarships:

First, the result may suggest that students need just a small prompt to encourage them to put more effort into their studies but that larger incentives are unnecessary. Further, it is possible that as the value of the incentive payment (external motivation) increases, students’ internal motivation declines at a faster rate such that negative impacts on intrinsic motivation increasingly moderate any positive impacts of the incentive on educational effort.

Not all incentives are monetary, and the intrinsic motivation to study hard is important. Extrinsic motivations can sometimes over-ride intrinsic motivations, creating negative unintended consequences. For one (famous) example, read this study by Uri Gneezy and Aldo Rustichini about incentives in Israeli day-care centres (also described in Gneezy's book co-authored with John List, The Why Axis, which I reviewed here). In that example, day-care centres began fining parents who showed up late to pick up their children, but this new system replaced the existing norm of picking up children on time, and actually resulted in more late pick-ups.

In the case of students, studying hard may be a norm for some students, and a small performance-based scholarship payment appears to reinforce the norm. However, if the payment is larger, then the intrinsic motivation to study hard may be eroded, leading to no further gains in study performance. People respond to incentives, but sometimes the incentives work in ways we don't anticipate. This is one area where more research is needed to fully understand why it is that the incentives don't scale with scholarship payments.

*****

[*] Alternatively, you could think of it as increasing the costs of not studying, since the student loses the scholarship if they fail to achieve the minimum performance standard. Regardless, the effect on incentives is the same - the student receiving a performance-based scholarship has an incentive to study harder.

No comments:

Post a Comment