Tuesday, 22 March 2022

Using microcommitments and social accountability to improve online and hybrid learning

One of the biggest challenges with online teaching is proving to be getting students to engage. It is much easier for students to take a passive (and much less effective) approach to their learning when they don't have to be present in class every week. The online approach generally fails to make students accountable for their learning, even though in theory the students should be more accountable to themselves, because it relies on their being more self-directed. But it needn't be that way.

This 2021 article by Amanda Felkey (Lake Forest College), Eva Dziadula (University of Notre Dame), Eric Chiang (Florida Atlantic University), and Jose Vazquez (University of Illinois at Urbana-Champaign), published in the AEA Papers and Proceedings (sorry, I don't see an ungated version) describes an experiment on increasing social accountability for students. Specifically:

...we conducted a randomized controlled experiment in fall 2019. Experiment participants were recruited from economics courses taught by six instructors at three universities, using three teaching modalities - face to face, online, and hybrid...

...students were randomly assigned to the control or treatment group. Between two exams in their courses, all participating students were sent daily content that corresponded to the material they were learning in their course. The content was designed to engage students for no more than five minutes per day. These small actions included problems, prompts to practice important concepts, and questions compelling students to relate course material to their own lives. No problems or answers were turned in by students or reviewed by instructors...

All students received the same daily content. Those in the control group received the content via text message, nudging them to do the small task. Students in the treatment group received the content from a platform, accessed via a link sent by text message, containing the same daily task but with a commitment device called a microcommitment. These students were asked whether they commit to doing the task that day. If they committed, the platform would follow up in the afternoon and ask whether they did the task... Commitments and completed tasks appeared on a social feed that provided social accountability.

So, what happens when students can see if the other students in their class are completing tasks? Felkey et al. find that:

An OLS analysis finds that microcommitments have a positive and significant effect on student performance... score. As a proportion of the class average, relative performance increased significantly by 0.015 to 0.017, or approximately 1.3 additional percentage points on the post-intervention exam for those in the treatment group. The positive effect of microcommitments was driven by improved performance among students in online and hybrid courses and is equivalent to approximately 3.5 additional percentage points on the post-intervention exam. The effect of microcommitments with social accountability on students in face-to-face courses is insignificant.

That the positive effect was concentrated on online and hybrid classes is important, given that the ongoing pandemic is ensuring a continuation of those teaching modes. Felkey et al. note that:

Perhaps this intervention with commitment devices and social accountability partially substitutes for the lack of instructor contact when a course is taught in an online or hybrid modality.

That seems sensible to me. If the problem with online learning is the lack of engagement, then perhaps the microcommitments and social accountability enforces a higher level of engagement among the online students. An important question, though, is which students benefit? As I've noted several times before, online learning can be good for self-directed high-ability students, but detrimental for low-ability students who lack a high degree of self-direction. Felkey et al. find that:

...microcommitments positively and significantly affect students with inferior previous academic performance as measured by reported GPA.

That might be the most important finding from this research. However, before we get carried away (and beyond just noting that this is a single study in a crowded literature):

While we would most like to reach students with the lowest self-efficacy with performance-enhancing interventions, microcommitments do not have a significant effect on performance among these students. However, they do positively affect performance among students with above-average self-efficacy.

Their measure of self-efficacy is based on the Motivated Strategies for Learning Questionnaire (MSLQ), which measures learning strategies and academic motivation. So, while the microcommitments and social accountability works for low-performing students, it doesn't work for students with low academic motivation. It would be nice to know how the interactions work here, but despite mentioning the results in the text, Felkey et al. don't report the results by GPA either in the paper or in the online appendix. Importantly, since lower-performing students and more motivated students both do better with the microcommitments, how to low-performing and unmotivated students perform? We don't get an answer to that.

Nevertheless, this paper provides us with some small hope that there are strategies that can be implemented to engage students in online learning and to mitigate some of the negative effects of learning in that environment, especially for low-performing students. The questions are, whether this extends to other contexts, and how it can be implemented in various learning management systems (like Moodle or Blackboard).

Read more:

at

No comments:

Post a Comment