Importantly, this paper was able to establish causal estimates of the impact of online delivery on student performance in the course and in subsequent courses, using an instrumental variable strategy. However, first it is worth noting that the particular setting is important:
Each course is offered both online and in-person, and each student enrolls in either an online section or an in-person section. Online and in-person sections are identical in most ways: both follow the same syllabus and use the same textbook; class sizes are approximately the same; both use the same assignments, quizzes, tests, and grading rubrics. The contrast between online and in-person sections is primarily the mode of communication. In online sections, all interaction—lecturing, class discussion, group projects—occurs in online discussion boards, and much of the professor’s lecturing role is replaced with standardized videos. In online sections, participation is often asynchronous while in-person sections meet on campus at scheduled times. In short, the university’s online classes attempt to replicate its traditional in-person classes, except that student-student and student-professor interactions are virtual and asynchronous.So, the only difference between the online and traditional in-person classes is that the online course is run online (with necessary differences in communication between students and the lecturer). To get at the causal estimates though, Bettinger et al. use the interaction between the distance to the nearest campus (of which there were over 100) and whether the course was offered both online and in-person in that semester. As they explain:
...in the interaction instrument design, the reduced-form coefficient only measures how the slope, between distance and grade, changes when students are offered an in-person class option. The main effect of distance (included in both the first and second stages) nets out any other plausible mechanisms which are constant across terms with and without an in-person option. Parallel reasoning can be constructed for the Offered component of the instrument.I know that's quite technical, but essentially Bettinger et al. are comparing the outcomes for students who did, and did not, take the online course when it was available to them, knowing that distance is a determinant of whether the students would take the in-person course. I hadn't seen this type of interaction instrument used before, and it's something that might come in handy in some of my other work.
The results that Bettinger et al. find were not surprising to me:
The estimated effect of taking a course online is a 0.44 grade point drop in course grade, approximately a 0.33 standard deviation decline. Put differently, students taking the course in-person earned roughly a B− grade (2.8) on average while their peers in online classes earned a C (2.4). Additionally, taking a course online reduces a student’s GPA the following term by 0.15 points. The negative effect of online course-taking occurs across the distribution of course grades. Taking a course online reduces the probability of earning an A or higher by 12.2 percentage points, a B or higher by 13.5 points, a C or higher by 10.1 points, and a D or higher (passing the course) by 8.5 points...Basically, it's all bad news for online courses. They also find that the impacts are greatest for those at the bottom of the grade distribution, with statistically insignificant effects on the top three deciles of students. That isn't too much different for the results on flipped classrooms that I have discussed before (see here and here). Students also do significantly worse in future courses (after the online course), and are significantly less likely to be enrolled in the future (they are more likely to drop out).
There are some important take-away messages from this research. First, many universities are progressing towards online delivery, either alongside traditional in-person delivery or in place of it. There may be cost savings to online delivery, but those cost savings need to be carefully weighed up against the worse student outcomes that can be expected from online courses. Second, many students seem to be very keen on signing up for online courses, in preference to in-person classes, probably because of the flexibility that an online course provides. Again, the benefit of flexibility needs to be carefully weighed up against the worse outcome (not just in that course, but in future courses) that would result from taking the online course.
Of course, as Bettinger et al. note in their paper, for some students there is no alternative to online education. However, for those who do have a choice, taking an in-person class may be preferable.
Read more:
- Online vs. blended vs. traditional classes
- Flipped classrooms work well for top students in economics
- Flipped classrooms are still best only for the top students
No comments:
Post a Comment