So, I was somewhat unsurprised by the results from this 2016 article by Gerald Eisenkopf (University of Konstanz) and Pascal Sulser (eBay), published in the Journal of Economic Education (it appears to be open access, but just in case there is an ungated earlier version here). Eisenkopf and Sulser ran an experiment in 29 Swiss upper secondary schools (Kantonsschule/Gymnasium), where classes were randomised between:
- Classes that employed the usual instructional methods with the usual textbook resources (the Control group);
- Classes that introduced the topic using a classroom experiment (the Experiment group); and
- Classes that introduced the topic through the teacher developing their own lecture material, but where experiments were not allowed (the Standard group).
In all cases, the evaluation was limited to the topic of common pool resources, and the experiment was remarkably similar to one that I run with my ECONS102 class each year:
At the beginning of the game, the pond contains four fish per player. In each of the 10 rounds, every player may catch between zero and three fish anonymously (by wearing masks). The number of fish remaining in the pond doubles between rounds. However, there is a capacity limit. The pond cannot hold more than four fish per player. Students are told that they win the game by catching the most fish.The incentive for each player is to catch the most fish (and thereby win the game). However, in doing so, the pond quickly runs out of fish and all players are left with an empty pond. That is the nature of the Tragedy of the Commons.
Anyway, how well did the students in each group do? In a test of understanding after the topic, Eisenkopf and Sulser found that:
Absent any treatment intervention, 58 percent of all statements were solved correctly, while roughly 30 percent were answered falsely. Hence, the Control group managed to achieve about 28 percent of the theoretical maximum score (4.7 score points out of 17). Students of both treatment groups fare much better, yielding average scores of 50.49 percent (8.58 points) in the Standard and 50.41 percent (8.57 points) in the Experiment treatment. Evidently, both teaching interventions were able to increase economic understanding considerably, with an effect size of about 0.8 of a standard deviation each... Aggregate scores between the Standard and the Experiment group are remarkably similar...So, students in either the Experiment or Standard group fared better at the end of the topic than students in the Control group. However, when they look at which students do better, by interacting the treatment effect with students' overall economic knowledge (based on a test of general economic understanding), they find that there are no differences in the Standard group, but that:
...our classroom experiment favors more competent students, while weaker students are worse off than they would be under a regime that depends on conventional teaching.Based on their discussion of the results, it seems that the classroom experiment crowded out time for reading case studies and working on practice exercises. This wasn't a problem for students who had a high level of economic understanding, presumably because they were able to make the links between the experiment and the concepts quickly. However, the less able students were disadvantaged because they had less opportunity to develop the understanding, which they may have done through more traditional methods.
Unfortunately, as this study demonstrates, sometimes the coolest methods of teaching are not the most effective.
Read more:
- A cautionary tale on analysing classroom experiments
- The diminishing marginal returns to classroom experiments
No comments:
Post a Comment