Wednesday 25 March 2020

Online classes may also make better students worse off

I've written a number of posts about various research papers looking at the impact of teaching online (see the links at the end of this post). One of the things I have taken away from that literature is that teaching online (or blended learning - a mix of online and face-to-face) appears to improve learning outcomes for highly motivated and engaged students, but makes things worse for less motivated or engaged students.

However, this 2016 article by Jennifer Heissel (Northwestern University), published in the journal Economics of Education Review (sorry, I don't see an ungated version online), makes that simple conclusion much less clear. Heissel investigates the effect of online teaching of Algebra I to relatively high performing eighth-graders in North Carolina. She makes use of an interesting natural experiment:
Columbus County Schools offer a potential natural experiment to exploit. Before 2011, CCS had no Algebra I option for their middle school students; instead, their advanced students received some supplemental learning as part of their regular eighth-grade math class. Beginning in 2011, CCS began offering NCVPS [North Carolina Virtual Public School] Algebra I to their advanced eighth graders. Classes of students met in their school computer lab and participated in the virtual Algebra I course. The virtual teacher was the primary instructor, but a local staff member supervised the class.
She compares the eighth graders who took the online Algebra I course with eighth graders who took Algebra I using a traditional (face-to-face) class, and with ninth graders. Since Algebra I is typically taken in ninth grade, eighth graders who take it tend to be near the top of the performance distribution for mathematics (and academically overall). Using data from the 2010/11 and 2011/12 academic years, for nearly 200,000 students (of which only 719 took Algebra I via NCVPS), she finds that:
...NCVPS students have between 0.28 and 0.31 standard deviations lower test scores than students in traditional classrooms. Passing rate differences are not consistently statistically significant at conventional levels...
When looking at the results by student mathematical ability, she finds that:
...the top-performing quintile in CCS had scores 0.30 standard deviations below expected after the policy change. The second-highest quintile had scores 0.24 standard deviations below expected...
The results are robust to various differences in econometric specification. She concludes that:
 Virtual students appear to underperform relative to similar peers in traditional middle school Algebra I classrooms and relative to students who wait until ninth grade to take the course. Policymakers should carefully weigh these tradeoffs.
The trade-offs here are the (supposedly) lower cost of teaching online vs. the lower academic performance of these students. Besides highlighting that trade-off, these results are interesting to me in particular because we teach some accelerated students in my ECONS101 class, through the UniSTART programme. That programme allows senior high school students to take university-level papers in an online format. Those students tend to perform quite well in ECONS101, and in the past these students have frequently dominated the ranks of the top ten students (and beyond) overall. However, the students who are allowed into the UniSTART programme tend to be those that are relatively high achievers. It makes me wonder - could they have done even better if they had waited to study on-campus?

In any case, this paper adds to the growing evidence that suggests to me that online teaching is not an optimal format. And that is highly relevant given the current lockdown, which has driven all of our university teaching online.

Read more:

No comments:

Post a Comment