People lead complicated lives. They have many competing goals, and have to trade off between those goals. Economists assume that they choose their actions with the overall aim of maximising their utility (satisfaction, or happiness). However, the many competing goals can sometimes thwart well-intentioned interventions. For example, when seatbelts were made compulsory, that made driving faster safer to do, and people responded by driving faster, and therefore less safely (for related examples, see here and here). Economists refer to that as offsetting behaviour.
Two recent examples of this arose in research papers I read this week. The first is this NBER Working Paper by Hunt Allcott (Stanford University) and a long list of co-authors, who investigated the impact of people temporarily deactivating Facebook or Instagram on their emotional state. Working with Meta (where some of the co-authors work), they:
...recruited 19,857 Facebook users and 15,585 Instagram users who spent at least 15 minutes per day on the respective platform. We randomly assigned 27 percent of participants to a treatment group that was offered payment for deactivating their accounts for the six weeks before the election. The remaining participants formed a control group that was paid to deactivate for just the first of those six weeks.
They then compare the difference in emotional state between before and after the deactivation for the treatment group (who deactivated for six weeks) and the control group (who deactivated for one week), and find that:
...users in the Facebook deactivation group reported a 0.060 standard deviation improvement in an index of happiness, anxiety, and depression, relative to control users...
...users in the Instagram deactivation group reported a 0.041 standard deviation improvement in the emotional state index relative to control.
Those effects are quite small in comparison to other interventions, and in comparison to changes in emotional state over time, and:
Under the approximation that emotional state index is normally distributed, the estimated effects of Facebook or Instagram deactivation would move the median user from the 50th percentile to the 52.4th or 51.6th percentile, respectively.
Why was the effect so small? Users who deactivated Facebook or Instagram spent more of their newly-freed-up time on other apps. Those who deactivate Facebook increased their use of Instagram, but also:
Facebook and Instagram deactivation both increased use of Twitter, Snapchat, TikTok, YouTube, web browsers, other social media apps, and other non-categorized apps by a few minutes per day.
It's little wonder that deactivating Facebook or Instagram had such small effects, given the offsetting behaviour of the users pivoting to using other apps, including other social media apps, instead. None of this is to say that the intervention made the users worse off, but it probably didn't make them better off overall either.
The second example is this NBER Working Paper by Billur Aksoy (Rensselaer Polytechnic Institute), Lester Lusher (University of Pittsburgh), and Scott Carrell (University of Texas at Austin), which looked at the effects of the app 'Pocket Points' at Texas A&M University. Specifically:
Pocket Points is marketed as a soft commitment device and provides incentives for students to stay off of their phones. In particular, Pocket Points rewards students with “points” for staying off their phones during class: Students open the app, lock their phone, and start accumulating points, all while the app verifies through GPS coordinates that the student is indeed in class. These points can then be used to get discounts at participating local and online businesses.
One thousand Texas A&M students were invited to participate in the experiment in 2017, and half were randomised to treatment, where they were instructed to download the Pocket Points App and create an account. Aksoy et al. then compare the treatment and control students. They also distinguish effects between those who used the app at least once, and those who used the app more than once a week (based on survey results). Importantly, first Aksoy et al. report that:
...treatment students were about 25 percentage points more likely to download the app... and over 31 percentage points more likely to use the app... than control students. Additionally, treatment students were 13 percentage points more likely to use the app more than once a week...
So, the treatment worked in encouraging students to use Pocket Points. But did it work? Aksoy et al. find some positive effects in the classroom, such as:
...Pocket Points usage is associated with a 0.42 standard deviation reduction in phone distraction rate in the classroom... we observe increases in student satisfaction with their academic performance for the semester: Students who used the app more than once a week experienced more than a one standard deviation increase in satisfaction...
That seems promising. However, when they look at student grades (from their official TAMU transcripts), Aksoy et al. find that:
...students who used the app more than once a week experienced a 0.50 unit increase in GPA. These estimates, however, are statistically insignificant...
So, even though the Pocket Points app reduced in-class distractions, it had no statistically significant effect on students' grades. That may be because there were also:
...significant decreases in time spent studying on campus... treated students spent approximately 18.2 hours/week studying, 12.0 of which were on campus, whereas control students spent 20.3 hours/week studying, 14.1 of which were on campus. Thus, it appears that the increased learning and attendance in the classroom came with a reduction in time spent studying.
It's little wonder that there was no effect on students' grades, given the offsetting behaviour of students spending less time studying, perhaps because they believed (perhaps rightly) that their in-class study time was more effective without phone distractions. None of this is to say that the app made the students worse off, but it probably didn't make them better off overall either.
When we implement an intervention that we hope will lead to better outcomes, such as improved emotional state due to less time spent on social media, or improved student performance due to more focused studying in class, we need to be prepared for the offsetting behaviour of the people affected by the intervention. Their lives are complicated, and they are trading off between competing goals. Just because we want to make one of their goals easier to achieve, that doesn't mean that they will focus extra energy on that goal. As we have seen from the two examples above, they may simply re-focus their energies elsewhere, leaving the outcome that we want to improve unchanged.
[HT: Marginal Revolution, last year]
No comments:
Post a Comment