The diminishing marginal returns to classroom experiments

I'm a big fan of using classroom experiments to illustrate concepts to my students. When I taught managerial economics, I specifically organised the workshops and assessment to incorporate many experiments. I use a number of experiments in ECON110, which are worth extra credit for students.

So I was interested to read this recent paper by Tisha Emerson and Linda English (both Baylor University), published in the Papers and Proceedings issue of the American Economic Review (sorry I don't see an ungated version anywhere). In the paper, Emerson and English look at 28 principles of microeconomics classes from Baylor between 2002 and 2013, some of which used experiments (between 6 and 11 experiments) and some of which had no experiments (and act as a control group). Looking at overall performance in the course, they find:

...a statistically significant positive relationship between the number of experiments in which a student participates and their course score. The impact is diminishing, however, as the number of experiments increases.

The impact of experiments is different for different groups:

While older students outperform younger ones, the youthful disadvantage is partially offset by participation in experiments. Similarly, our findings suggest that an achievement gap exists between whites and ethnic minorities, but that experiments serve to help bridge this gap as well. However, experiments do not appear to differentially impact student performance by gender, aptitude, or attendance.

So, the positive effect of classroom experiments is largest for the first experiment, and then diminishes. We can use the coefficients from Table 2 in the paper (shown below) to calculate the optimal number of experiments (see [*] for one example of the calculus if you're feeling adventurous) - that is, the number of classroom experiments that would maximise grades for a student with a given set of characteristics. The optimum differs by group (because of the interactions between the number of experiments and gender, age, nonwhite, SAT total, and absences from class).

For a 19 year old white male (with the mean SAT of 1171 and the mean absences of 2.59), the optimal number of classroom experiments is 5.3, while for a 19 year old nonwhite male (with mean SAT and absences) the optimum is 7.5, and for a 19 year old nonwhite female (with mean SAT and absences) the optimum is 8.0. Interestingly, the optima are much lower for older students. For example, for a 23 year old white male (with mean SAT and absences), the optimum is -0.4. So, most of the gains from classroom experiments in this sample accrue to younger and nonwhite students.

Anyway, overall this tells us that classroom experiments are likely a good thing, and are likely an even better thing for traditionally disadvantaged groups. Indeed, the authors conclude:

...experiments may be providing market experiences that these groups (younger and minority students) may not otherwise have had and thus greater understanding of economic concepts that can’t be gleaned simply from class discussion. As such, the use of classroom experiments may help reach these otherwise disadvantaged groups.

So I guess I should persist with classroom experiments, just not too many!

*****

[*] Using the coefficients for a 19 year old white male (with the mean SAT of 1171 and mean absences of 2.59):

G =  (9.948 - 0.171 - 0.444 * 19 - 0.000186 * 1171 + 0.201 * 2.59) x - 0.154 x^2 + z
    = 1.644 x - 0.154 x^2 + z

where G is the student grade (percentage), x is the number of experiments, and z is all of the other parts of the regression equation that don't include the number of experiments.

Differentiate the function G with respect to x:

dG/dx = 1.644 - 0.308 x

Set this equal to zero, and solve for x:

1.644 - 0.308 x = 0
x = 5.3

So, the percentage grade is maximised for students with these characteristics when they participate in 5.3 experiments. Similar calculations can be made for other combinations of characteristics.