The gender bias in student evaluations of teaching is well established (see my most recent post on the topic here, or the links at the end of this post). However, during the pandemic teaching underwent a sudden and unexpected change to an online mode. It is reasonable to ask, has online teaching reduced gender bias in evaluations? On the one hand, we know that women faced additional difficulties in managing the transition to online work, and especially juggling work with unequal home and family responsibilities. This may have impacted on female teachers' ability to deliver teaching in the online mode effectively. On the other hand, at the risk of gross generalisation, women tend to have a different teaching style that favours connections over content, which may have better helped students with the transition to online learning. It is therefore unclear whether female teachers would be helped, or hurt, in terms of student evaluations of teaching, by the move to online teaching.
This new article by Sara Ayllón, published in the journal Economics of Education Review (ungated earlier version here), provides us with an initial answer. Ayllón uses data from the University of Girona in Spain. Using a difference-in-differences approach, she compares the difference in teaching evaluations between the first and second semester of the 2018/19 academic year, with the difference in evaluations between the first and second semester of the 2019/20 academic year. Since online teaching was enforced in Spain for much of the second semester of 2019/20, this comparison is intended to pick up the impact of online teaching on evaluations. In her baseline analysis, she finds that:
...when I use controls (student’s age, its square, gender, whether the student is repeating the course, and field of study) and robust standard errors clustered at the student level... teaching evaluations during the online semester were, on average, no different from those in previous semesters. Interestingly, though, separate regressions by gender of the lecturer indicate a different story... the online semester had, on average, no impact on the evaluation of male lecturers; but for female lecturers, the average evaluation score decreased by 0.063 points in the online semester compared to previous semesters (about 5.4% of a standard deviation). Thus, while the new teaching environment had, on average, no effect on men’s scores, it did negatively impact the scores received by women...
Ayllón then attempts to tease out the reasons underlying the negative impact of online teaching on female lecturers' student evaluations. She finds no difference in how students felt about how well the course materials were adapted to online learning between male and female lecturers. She also finds no robust difference in grades between students with female lecturers and students with male lecturers. And there is no difference in students' opinions on various aspects of lecturer performance. Ayllón notes that:
...the gendered difference in the teaching evaluation result of the online environment does not appear to be driven by (potentially more objective) aspects of the teacher’s performance. The bias creeps in when students evaluate overall performance...
Students couldn't have easily sorted themselves to have different lecturers, because they chose their programme of study at the start of the academic year. Nevertheless, Ayllón shows that the results hold when the sample is limited to compulsory courses. Finally, looking more deeply at the characteristics of lecturers and students, she finds that:
...the results are particularly negative for young female instructors without a permanent contract, and are strongly driven by male students and low achievers who - even before they know their final grade - retaliate against female instructors, but not against male teachers. The findings are most apparent in Social Sciences. Online teaching did not lead to any positive bias on the part of female students towards female instructors. Yet a considerable degree of discrimination in favour of male instructors is found among high-achieving students.
These results are not dissimilar to other results in the literature. However, rather than showing the underlying gender bias in teaching evaluations, this study shows that the gender bias is larger during online teaching. Unfortunately, as with most studies like this, the solution to the problem is unclear. With online teaching, female lecturers can't give their student evaluations a bump by giving students chocolate. However, these results, especially if confirmed in other studies, should make us even more cautious about using data from student evaluations of teaching in promotion or tenure or appointment decisions, unless we want to perpetuate gender bias.
Read more:
- Gender biases in student evaluations of teaching
- Are student evaluations of teaching even measuring teaching quality?
- Gender bias in principals' evaluations of primary teachers in Ghana
- More on gender bias in student evaluations of teaching
- Can a simple intervention reduce gender bias in student evaluations of teaching?
- Hardly the final words on student evaluations of teaching
No comments:
Post a Comment