Universities are quite focused on student retention, and as I noted in this 2018 post, if we can identify at-risk students, perhaps we can help to find ways to ensure they succeed. However, around that time I had a summer research scholarship student looking into the reasons that students drop out, and it turned out that each student dropped out for quite different, and difficult to predict, reasons. I call this the Anna Karenina principle of dropout: 'all students who persist are alike; each student who drops out does so in their own way'.
What if there were a simpler way of reducing student dropout, that did not require universities to identify at-risk students in advance, but instead reduced the risk of dropout from the outset? That would seem to be an attractive proposition.
So, I was interested to read this 2021 article by Lucio Masserini (University of Pisa) and Matilde Bini (European University of Rome), published in the journal Socio-Economic Planning Sciences (ungated version here), which evaluates the impact of student-created social media groups, such as Facebook pages, on student dropout. Masserini and Bini use survey data from 1879 first-year students from a major university in Central Italy.
Why would joining social media groups reduce dropout? Masserini and Bini suggest that these groups may help students form social connections and feel greater 'belonging' within the university community, while also providing a way for students to share information about courses, assessments, and study materials.
The key challenge in the analysis is that students are not randomly assigned to join social media groups - they choose whether or not to do so. And students who join these groups may differ from those who don't in ways that would bias a simple comparison of the students who joined social media groups and those who didn't. For example, more engaged students, who are less likely to drop out, might also be more inclined to join university-related social media groups run by other students. Masserini and Bini deal with this using propensity score matching - which involves identifying 'control' students who didn't join a social media group but who are most similar to each 'treated' student who did join a social media group. Then, comparing their matched control and treated students deals with any observable differences between the students who did, and did not, join social media groups.
Masserini and Bini then report a range of results of the estimated impact of social media groups on dropout, based on different assumptions used to do the matching, and:
...with the exception of k=1 nearest-neighbour, all the estimates indicated that students joining groups or Facebook pages had, on average, a lower probability to dropout, compared with those who were not part of such groups. The results also showed that the extent of the difference between the treated and control groups was not negligible, as it varied from 0.081 to 0.113, depending on the matching algorithm.
So, the results suggest that joining student-run social media groups or Facebook pages reduces the probability of a student dropping out by between 8.1 and 11.3 percentage points. Now, I should note that I don't in general find propensity score matching to be terribly convincing as a way of dealing with selection bias.
Now, I should note that I do not find propensity-score matching entirely convincing as a way of dealing with selection bias. Although matching can make the treatment and control groups similar on observed characteristics, there is still something that is different about the treated and control students that leads the treated students to choose to join social media groups and the control students to choose not to join. That something is an omitted variable in the propensity score matching approach, and it is unclear how big the omitted variable bias will be. If, for example, joiners are more motivated or feel more connected to university life, then some of the apparent effect of joining the group on the probability of dropping out may instead reflect those underlying differences. Masserini and Bini's results are robust across several matching methods and sensitivity checks, which is reassuring, but robustness checks cannot establish that there isn't some omitted variable bias in the matching.
Having said that, if we take these results at face value, then there may be some merit in having student-run social media groups that university students can join. We must bear in mind that these results come from a survey in 2016, and they may not have aged well. But social media groups still exist, and students still participate in them. It could be worth exploring whether these effects still hold, given that increasing student retention remains a key focus for universities.
Having said that, student-run social media groups would probably be a relatively inexpensive way for universities to reduce dropout. Now, these results come from students surveyed in 2016, and both social-media use and the university environment have changed considerably since then. Nevertheless, the basic idea remains plausible. Universities could support the creation of student-run groups and randomly encourage or 'nudge' some students to join, then compare their subsequent retention with that of students who were not encouraged. That experimental approach would provide more contemporary and causal evidence of whether the groups reduce dropout, rather than merely attracting students who were already less likely to drop out.
Read more:
No comments:
Post a Comment