We introduced the CORE textbook The Economy at the University of Waikato when we recoded the compulsory economics paper in our management degree from ECON100 to ECONS101 (see here). We were early adopters, as the CORE textbook was only released in 2017. It was a big change, and largely a positive one. The CORE textbook was free, substantially lowering the cost for students to access an important learning resource. Because the textbook was online, it could be constantly updated. And I really liked the way that it turned the traditional approach to the teaching of microeconomics on its head. Instead of starting with perfect competition and the supply and demand model, and then teaching imperfect competition as an exception, the CORE textbook started with monopolistic competition (where firms sell products that are differentiated from those of their competitors) and teaches perfect competition as an exception. Since many firms operate in monopolistically competitive markets, the approach that CORE adopted seems more attuned to the real world that students see.
I've often wondered whether the CORE textbook improved students' learning though. So, I was interested to read this recent article by Federico Araya (Universidad de la República, Uruguay) and co-authors, published in the journal Economica (sorry, I don't see an ungated version online [*]). They evaluate the impact of adopting the CORE textbook for the introductory economics course at the Faculty of Economic Sciences and Administration (FCEA) at the Universidad de la República, the largest university in Uruguay.
Although the CORE textbook was introduced at FCEA in 2020, Araya et al. start their analysis from 2021, to avoid the impacts on online teaching during the pandemic. FCEA offered two introductory microeconomics courses, one of which used CORE and the other continued to use their traditional textbook. Students were randomly assigned to either course based on the last number of their identification document, with 30 percent of students assigned to the course that used CORE.
However, the textbook was not the only difference between the two courses. As Araya et al. explain:
Although attendance is optional in both courses, in 2022 and 2023, the CORE course introduced a modification to its evaluation system, assigning 10% of the total grade to group activities conducted during class sessions. This change may have created an incentive for higher attendance.
So, their evaluation is not a clean comparison of the same course taught with two different textbooks, but will compare two different pedagogies, one which uses the CORE textbook and in-class group activities that are worth grade points, and one that uses a traditional textbook without the in-class group activities.
Araya et al. then compare the two groups in terms of whether students passed the introductory microeconomics course, as well as whether students passed an introductory calculus course and whether they passed the intermediate microeconomics course that follows on from the introductory course, while controlling for a range of demographic and socioeconomic variables for each student. They find:
...no statistically significant differences in pass rates between CORE and the conventional course, with the exception of the 2022 cohort.
I was initially surprised that they decided to evaluate each cohort separately, rather than pooling them. However, the 2021 cohort is different because that year the CORE course didn't have the in-class group activities, whereas it did for the 2022 and 2023 cohorts. Combining the 2022 and 2023 results would give us a better sense of the overall effect (of the combined CORE textbook plus group activities intervention). Instead, we are shown a statistically significant positive effect in 2022, but no statistically significant effect in 2023. That doesn't tell us whether the effects in those two years were actually different from each other, or whether the combined intervention had a positive overall effect across the two cohorts. One further problem here that muddies the comparison is that the CORE and traditional courses didn't use the same assessment, and so passing one course may be different from passing the other. And that might also explain the different cohort-specific results (if the 2022 traditional course had more difficult assessments than the CORE course, for example).
That won't be a problem for comparisons in terms of student performance in introductory calculus and intermediate microeconomics. For those courses, Araya et al. also find no statistically significant effects on passing.
So, at least there is no evidence from this study that the CORE textbook (with or without in-class group activities) made students worse off. Although equally, there is no evidence that it made them better off either. That allows me to raise an issue that is general to much of the similar research on educational interventions (including my own research on the impact of AI tutors in ECONS101). We might expect to see no significant effect on student performance even from a successful intervention. That's because a successful intervention may make studying easier for students, freeing up time that they can then devote to other activities. That might be studying for their other courses (although notice that in this case any reallocation of study effort doesn't appear to have affected the probability of passing introductory calculus), or something entirely different (maybe working more, or having more leisure time). So, I'm not surprised to see no effect of CORE on student performance in this study.
We continue to use the CORE textbook in my ECONS101 class (although last year we moved to the new edition, The Economy 2.0). We don't follow the text very closely, at least not in the microeconomics section of the paper that I teach. Nevertheless, it continues to provide the base material for a lot of what we teach. And it's good to know that at least one study says that there is not evidence that continuing to use a 'non-traditional' text is doing harm to students.
*****
[*] It's kind of ironic that a paper evaluating the impact of an open-access teaching resource is not itself published open-access.
Read more:
No comments:
Post a Comment