The research article that I referenced in my post yesterday pointed me to this 2019 article by David van Alten, Chris Phielix, Jeroen Janssen, and Liesbeth Kester (all Utrecht University), published in the journal Education Research Review (ungated version here). They report on a meta-analysis on the effect of 'flipping the classroom' on student outcomes. Flipping the classroom is an approach where, as van Alten et al. note:
...students study instructional material before class (e.g., by watching online lectures) and apply the learning material during class.
Under this definition, 'instructional material' might include online instructional videos, computer-based tasks, or simply reading material. This broader definition is problematic if what you really want to know is whether new technologies are having a positive impact on learning. It turns out the definition isn't a problem in this paper, because 110 of the 115 interventions (reported across 114 research articles) included in the review used online videos as the before-class component, so there is likely to be little bias arising from the inclusion of other modes. All included studies were published in the period from 1990 to 2016.
Van Alten et al. focused on three outcomes: (1) assessed learning; (2) perceived learning; and (3) student satisfaction. The difference between assessed and perceived learning is that assessed learning is based on the quantitative results of an assessment exercise (e.g. an exam, or a standardised test), while perceived learning is based on how much the students thought they learned. They found that:
The average effect size for assessed learning outcomes (g=0.36) was found to be significant... The average effect on perceived learning outcomes was nearly identical (g=0.36), but not significant (p=.13)... For student satisfaction, a trivial and non-significant effect size (g=0.05) was found.
It is interesting that students were not satisfied with the flipped classroom approach on the whole. The perceived learning results were pretty noisy, but the large positive effect on assessed learning outcomes doesn't really accord with my understanding of the literature to date (see the links at the end of this post). Van Alten et al. dug a bit deeper into their results by looking at the characteristics of the studies, and in terms of assessed learning outcomes they found that:
...studies that shortened the classroom time of the flipped condition had a significantly lower (p=.027) average effect than studies in which the classroom time in both conditions was equal (with a difference of g=−0.26, while accounting for all the other variables in the model). In addition, adding quizzes in the flipped condition also showed a significant (p=.044) difference with studies where quizzes were not added or already applied in the traditional condition (with a difference of g=0.19, while accounting for all the other variables in the model).
In other words, the positive outcomes from flipping the classroom arise when the before-class tasks are additional to the classroom learning time. It is encouraging the students to do additional work, and that might be driving the effects, rather than the flipping of the classroom on its own. That also flies in the face of one of the rationales for flipping the classroom, that it reduces the time commitment required of teaching staff. If the same number of contact hours are required, along with the preparation of videos as well, then that requires more time.
There are some problems with the meta-analysis, and one arises from the nature of the included studies. Only about a quarter of the studies employed randomisation, and so the other studies lack a bit of experimental rigour. It would have been interesting to see what the results look like when limited only to the randomised studies (although, to be fair, randomisation wasn't statistically significant in the meta-regression, so perhaps it wouldn't make much difference).
The other issue is heterogeneity, which is an issue I have raised several times before (e.g. see here or here). Blended learning works well for highly-engaged high-achieving students, but it could well be net negative for less-engaged low-achieving students. It would be nice to see one of these meta-analyses or systematic reviews of the literature engage with a really important issue. On a related note, it would be interesting to see the meta-regression results of these analyses stratified by the average SAT scores of the samples (or similar measure of student ability). It could be that the studies that find the most positive effects tend to be undertaken in universities where the student body is (on average) higher ability and more engaged. Until we see some analysis along these lines, my critique of blended learning and flipped classrooms (and online approaches more generally) remains live.
Read more:
- Online vs. blended vs. traditional classes
- Flipped classrooms work well for top students in economics
- Flipped classrooms are still best only for the top students
- Online classes lower student grades and completion
- Online classes may also make better students worse off
- Meta-analytic results support some positive effects of using videos on student learning
- New review evidence on recorded lectures in mathematics and student achievement
No comments:
Post a Comment