Different universities employ different academic calendars. Many universities have a semester calendar. Other universities have a trimester calendar (the University of Waikato fits somewhat in-between these two, with two normal-length trimesters, and a summer trimester that is slightly shorter). Still other universities (although none in New Zealand) employ a calendar of four terms (or quarters). Which option is best for student learning? Some recent research may help us get part of the way to answering that question.
This new article by Valerie Bostwick (Kansas State University), Stefanie Fischer (Monash University), and Matthew Lang (University of California, Riverside), published in the American Economic Journal: Economic Policy (ungated earlier version here) compares outcomes between semesters and quarters. Bostwick et al. first note the theoretical ambiguity in which system is best:
A priori, the effects of the calendar system on student outcomes are ambiguous. A semester calendar has longer terms, requires one to take more courses per term to remain a full-time student, and operates over a different set of months than a quarter calendar. As such, semesters may be more conducive to learning and/or degree attainment, as there is a longer time horizon to master complex material. They may also provide more summer internship opportunities due to their earlier end dates in the spring term. On the other hand, it is possible that the longer terms unique to semesters may allow one to become complacent or procrastinate between exams, leading to poorer performance. Moreover, the greater number of simultaneous courses in a semester term may be difficult to juggle and/or pose scheduling challenges.
Bostwick et al. then use two datasets to investigate the differences in student outcomes between a semester academic calendar and a quarters academic calendar. The first dataset covers nearly all non-profit four-year colleges and universities in the US, covering all cohorts of students that began their studies between 1991 and 2010 (except for the 1994 cohort, where graduation rates were not available in the dataset). Their data has nearly 14,000 annual observations, from over 700 institutions. They then make use of the fact that some universities changed their academic calendar from quarters to semesters, and run both an event study analysis and a difference-in-differences analysis. Both analyses essentially compare the difference in four-year and six-year graduation rates between the period before and after the change in calendar, between universities that changed to semesters and those that did not.
Their results from this first analysis are neatly summarised in Figure 2 from the paper:
The years from -9 to -4 are the time before the shift to semesters for the treated universities. In the years from -3 to -1, students did part of their studies under the quarters calendar and part under the semesters calendar (the universities are 'partially treated'), while in the years from 0 onwards, all students did all of their studies under the semesters calendar. Notice in the top panel (four-year graduation rates), there is a clear downward shift once universities make the shift to semesters (fewer students graduate within four years). In the bottom panel (six-year graduation rates), the effect is smaller and not statistically significant.
Bostwick et al. then go on to look at individual student-level data from the Ohio university system, which allows them to explore the mechanisms that underlie the effects observed at the university level. Specifically, they have data from all students enrolled at 37 campuses for the first time between 1999 and 2015 (over 700,000 observations). Using these data allows them to look at a greater range of student outcomes, including graduation rates, credit workload, grades, and whether they switch major. They also link the student data to employment data from the Ohio Department of Job and Family Services, which allows them to look at in-semester employment and summer internships (which they proxy by summer employment not in the retail or food services industries). The analysis approach is similar to that employed at the university level (event study, and difference-in-differences), which is supported by variation across campuses:
There are 16 campuses in the data that were already on a semester calendar at the start of the sample in 1999. Four campuses switched from a quarter calendar to semesters over the course of the following decade. All of the remaining campuses in the state switched to a semester calendar in the fall of 2012 by mandate of the Ohio Department of Higher Education...
In their student-level analysis, Bostwick et al. first establish that the university-level results (lower rates of successful graduation within four years, or five years) still hold. They then explore the mechanisms for these results noting that:
There are a number of potential channels that could drive dropping-out behavior and/or an increase in time to degree. First, students and advisors may have difficulty navigating the transition to a new calendar system. We rule out this proposed channel because the estimated effects... are clearly evident in the long term...
A second potential channel is that students may find it challenging to juggle more simultaneous courses per term, as is required with a semester calendar. If this is a primary channel, students may earn lower grades or underenroll - that is, take fewer credits per term than what constitutes a full load... Lower grades could lead to an increase in time to degree if students are retaking courses for a better grade. Furthermore, if a student’s grades are low enough, they may face academic probation and potential dismissal from the university.
Finally, reduced scheduling flexibility associated with semesters caused by the longer-term length and higher number of required courses per term may be an important channel. Students might opt to take fewer courses per term to avoid unappealing class times (e.g., early morning classes)... It is also possible that scheduling flexibility impacts the timing and/or likelihood that a student switches majors, as major exploration is more costly under a semester calendar. Students who take longer to settle on a major are likely to experience a longer time to degree.
Bostwick et al. investigate all of these potential mechanisms, finding some weak evidence for a reduction in taking a full course load, but stronger evidence of a reduction in GPA (and increased risk of academic probation) and lower probability of switching major. Discussing this mechanism analysis, they note that (emphasis is theirs):
First, the higher number of courses per term may produce several of our findings. Students may find it difficult to balance more courses and topics simultaneously. This could explain the increase in the probability of falling below the 2.0 GPA cutoff. At the same time, some students may simply enroll in fewer credits per term (i.e., four courses instead of five) to avoid taking too many different courses at once... It is also possible that the higher number of courses in a term presents more of a scheduling challenge, particularly if a student wishes to avoid class times outside of the standard 9-5 school day...
Second, the increased length of the term may be at play. Longer terms could incentivize procrastination. There are longer periods between exams and more time to put off studying. It is possible that this type of behavior leads to lower grades and an increased probability of earning a GPA below a 2.0...
Additionally, longer/fewer terms mean that experimenting with a major takes more time. If, for instance, there are a set number of courses needed to learn about the match between one’s skills/interests and major, then this learning is more costly in a semester calendar, as one must commit to at least half a year in a major, compared to only a third of the year in a quarter system. Our findings on the timing of major switching are consistent with this proposed mechanism: students are no less likely to switch majors overall, but they are doing so later on in their college careers.
Finally, Bostwick et al. look at employment outcomes, and find that:
This analysis does not provide compelling evidence that the switch to a semester calendar improves summer employment in the types of jobs that are most likely to represent internship employment...
There is also some evidence of reduced in-semester employment, consistent with higher study workload. Overall, this study provides some compelling evidence that the shift from quarters to semesters has negative impacts on students. The disappointing thing is that there was no analysis of universities that have shifted in the opposite direction (from semesters to quarters), because very few have done so. Also, there is little guidance for universities with a trimester system, which were included in the quarters sample (although the analysis is apparently robust to changes in how those universities are treated).
Now, if we should prefer quarters over semesters, it is worth considering how far universities should shift in that direction. MBA or other executive education classes, for example, are often taught as block courses, over just a few weeks (I have taught on courses like this in the past, both at Waikato and through the New Zealand Institute of Highway Technology).
Victoria University in Melbourne recently moved to a block course format for their first year, and that change was investigated in this recent report (which was also discussed on The Conversation). As part of the change, they abolished traditional lectures, and moved to a format that emphasised smaller class groups. Unfortunately, the analysis in the report lacks the thoroughness and analytical rigour of Bostwick et al., being based mostly on qualitative data drawn from surveys with a small number of staff, academic leadership, and students. The small amount of quantitative results that are presented lack any statistical tests, so we are left to guess how meaningful the changes are. Nevertheless, the study does provide some interesting results that should encourage further exploration of this model, particularly:
...fail grades had dropped by 9.2 percentage points from the most recent pre-Block (2017) to post-Block (2019) cohorts, dropping 9.8 percentage points for equity students...
From 2017 to 2018, pass rates increased by 9 per cent for students in the highest socioeconomic status (SES) group, and 15 per cent for those in the lowest. Pass rates for students who were first in their family to attend university increased by 13 per cent, compared to 11 per cent for those who were not...
Now, pass rates can change for a number of reasons, so are not necessarily a reliable indicator of student learning. However, the bigger impacts on pass rates for students from lower socioeconomic groups and those who are first in family to attend university are encouraging (since these are groups that face substantial challenges in their transition to university study). The qualitative analysis also identified that:
Relationships between teaching staff and students were enhanced by the small Block classes...
Of course, small-group learning is not entirely absent in traditional university models, and some noted its similarity with tutorials... or Foundation Studies classes, which typically have small numbers of students... The difference with the Block was that academics were also immersed in the small-group format, for intensive periods with the same students.
Besides making learning more “enjoyable”... the closer relationship has positioned staff and students as partners in the learning experience.
Of course, not all of the positive improvement in staff-student relationships comes from the block format per se, since the smaller class size will facilitate that as well. However, anything that breaks down the divide between students and staff is a good thing (and is something that I work hard on in my large classes, but is much easier in a smaller class) and will help create a more welcoming environment. On that note as well:
The transfer of energy and enthusiasm from teaching staff to students was a clear, simple factor in the Block Model’s success. For equity group students, the teaching staff embodied a highly motivating university environment that welcomes, supports and believes in them.
The main (perhaps only) negative thing that comes out of the report is the effect on workloads. Students reported higher satisfaction with their classes in all aspects except for workload, and for staff:
...the intensive structure places high demands on staff workloads, especially in delivering assessment results in an extremely short timeframe...
So, the block model sounds interesting, and may have positive effects on students, particularly for equity groups. It is definitely worth exploring further, with a more rigorous study of its effects. Overall, it is clear that the academic calendar is an important consideration for universities, and along with the role of online learning, should be considered carefully in the future.
No comments:
Post a Comment