Columbia University students are willing to pay less for online tuition than for studying in person

Going into lockdown forced university teaching online. We heard a lot about how students were unhappy with online learning (e.g. see here and here). Students felt shortchanged by the new learning environment. So, if students prefer studying in person, we would expect them to be willing to pay for in-person classes compared with online study.

A new working paper by Zafar Zafari (University of Maryland), Lee Goldman, Katia Kovrizhkin, and Peter Muennig (all Columbia University), looks at exactly that question. They surveyed 46 Columbia University public health students, and the study had two interesting parts to it. First, they asked students to trade off the risk of becoming infected by coronavirus against attending classes in person. Second, they asked students how much they were willing to pay for online classes, in comparison with face-to-face classes. They found that:

On average, students were willing to accept a 23% (SE = 4%) risk of infection on campus over the semester in exchange for the opportunity to attend class in-person. Of the 46 students, 37 (80%) were willing to accept a >1% chance of infection and 3 (7%) were willing to accept a 100% chance of infection. One student was not willing to attend classes in-person unless the risk was 0%, and 9 (20%) were willing to attend in-person classes if the risk was less than 1%.

With respect to costs, students were willing-to-pay an average of only 48% (SE: 3%) of their tuition if courses were held exclusively online. No student was willing to pay full price for exclusively on-line instruction, and the maximum reported willingness-to-pay for online-only courses was 85% of standard tuition.

In other words, students in this sample are willing to accept a fairly high risk of coronavirus infection in exchange for attending classes in person, and they're willing to pay much less for online studying (and, by extension, willing to pay much more for the opportunity to attend classes in person). Of course, this should not be the last word on this topic. It was a study of just 46 students, and the methods are not what I would have used.

In fact, I wouldn't read much at all into the willingness-to-pay results - they simply asked students what they were willing to pay, which we know will be biased downwards (people will always say they are willing to pay less than they actually are, if only just in case they are later asked to actually pay!). They also anchored the willingness to pay by giving students a value first, then asking them what they would be willing to pay. It should be a surprise that the average result is about half of what they started with. It seems to me that a student, not knowing how much they would actually be willing to pay but knowing for sure that they wouldn't want to pay the full price, is likely to choose half price. And that's what they did.

A contingent valuation approach or a discrete choice experiment, where student respondents were asked to choose across a range of scenarios incorporating different levels of coronavirus risk, tuition costs, and whether studying was online or in person (and maybe other factors such as class size), would lead to much more plausible and defendable results. Hopefully, someone else is doing research along those lines.

[HT: Marginal Revolution]

Air pollution and the clean democracy hypothesis

Following on from Tuesday's post on the pollution haven hypothesis, I read this new article by Andreas Kammerlander and Günther Schulze (both University of Freiburg), published in the European Journal of Political Economy (sorry, I don't see an ungated version online). The paper tests the pollution haven hypothesis, but its main focus is the 'cleaner democracy hypothesis' - the idea that democracies are better at protecting the environment. As Kammerlander and Schulze explain:

According to this theory, there are five related causal mechanisms through which democracy leads to a cleaner environment: First, democracies allow a freer flow of information, and, therefore, environmental lobby groups are more effective in informing the population and raising awareness than in autocracies that censor information. Second, democracies protect the rights of civil society through freedom of speech and freedom of association, which makes it easier for environmental interest groups to organize and exert influence on the political process. Third, democracies are more responsive to demands of the electorate as incumbents are more accountable through free elections and environmental interests can seek political representation... Fourth, democracies are more cooperative and tend to honor environmental agreements as they are bound by the rule of law... Fifth, the members of the ruling elite in autocracies are less inclined towards environmental protection than the democratic public as they have a bigger share in the national income and the costs of environmental protection would therefore be higher for them...

They use cross-country data for 137 countries over the period from 1970 to 2012, to test whether there are robust correlations between measures of democracy and ten different pollutants:

These pollutants can be separated into gaseous air pollutants and aerosols. The available gaseous air pollutants are carbon monoxide (CO), nitrogen oxide (NOx), sulfur dioxide (SO2), non-methane volatile organic compounds (NMVOC), and ammonia (NH3). The aerosols in the dataset are black carbon (BC), organic carbon (OC), fine particle matter smaller than 10 μm (PM10), and fine particulates smaller than 2.5 μm (PM2.5), which are further differentiated between those originating from burning fossils and those from organic matter (PM2.5_fossil and PM2.5_bio).

If the cleaner democracy hypothesis is true, then you would expect to find similar effects for all pollutants, or at least not wildly divergent effects. Unfortunately, the news is not good for the hypothesis:

We find no consistent effect of democracy on pollution levels, neither in the regressions with country and time FE [Fixed Effects], nor in the regressions with only time FE. We do not even find that democracy has a consistent positive effect on the environmental quality for richer countries (as well-off citizens could have been hypothesized to be more likely to demand environmental control).

The results are robust (in the sense that alternatives also don't show any discernible pattern) to a battery of different specifications and the inclusions of different variables. They also show results that don't support the pollution haven hypothesis, unlike the paper I discussed on Tuesday, even with the use of various different measures of globalisation and trade intensity.

The problem with this analysis is that there is likely a lot of endogeneity in the regression models. Democracy might have causal effects on pollution, but there might also be other factors that cause both increases in democratic institutions and lower pollution - for instance, state capacity, social capital, or simply cultural differences in preferences for political institutions might be correlated with preferences for environmental quality. Also, there is likely to be a fair degree of multicollinearity in the model, as many of the variables will be correlated with each other. In the pollution haven hypothesis paper I discussed on Tuesday, Faqin Lin used an instrumental variables approach to overcome some of these issues, but that seems more difficult here, unless an instrument that affects democracy but not pollution (or preferences for environmental quality) could be identified. It is challenging but not impossible - this 2003 paper uses religion and socialist tradition, among other variables, as instruments for democracy. Having said that, I think it will be quite challenging, even with a better econometric approach, to revive the cleaner democracy hypothesis.

Read more:

• Trade openness, air pollution, and the pollution haven hypothesis

Meta-analytic results support some positive effects of using videos on student learning

The coronavirus pandemic and associated lockdowns forced teaching online. Teachers at all levels had to adapt their teaching to online delivery basically overnight. For most, that included both asynchronous recorded video 'lectures' that students watch in their own time, or synchronous video classrooms or workshops using videoconferencing tools like Zoom or Teams. Since the lockdowns have been relaxed, many teachers have continued to use videos in their teaching (in many cases, including at my institution, this was forced on teachers). A reasonable question, then, is what impact the use of videos has on student learning.

A new working paper by Michael Noetel (Australian Catholic University) and co-authors provides a fairly thorough answer, in terms of the impact of asynchronous, pre-recorded video content (there is also a non-technical summary of the research available on The Conversation). I say that this was a thorough answer because the authors conducted a meta-analysis of 105 different studies, with a combined 7,776 student research participants. Meta-analysis involves combining the results of many studies quantitatively, in order to determine more precisely any underlying relationship, and if it is executed well it can take account of publication bias (such as when statistically significant results are more likely to be published than statistically insignificant results). An added mark of quality in this particular meta-analysis is that they limited the included studies to randomised trials, which are more likely to establish causal estimates than observational or quasi-experimental studies.

The 105 studies included in the meta-analysis all compared the impact of either replacing face-to-face classes with video, or adding video to face-to-face classes, on student learning, measured as differences in grades or examination results or some other measure of academic performance (not differences in subjective measures, such as student evaluations of their own learning). All were conducted in higher education settings.

There are a lot of important results to unpack in this paper. First, I'll focus on the results that relate to replacing content with videos. The headline result is that:

...replacing other teaching with video had a significant positive effect on student learning...

I guess, in spite of my scepticism on the impacts of online learning (see this post, and the links at the bottom of this post), maybe there are advantages to it. In terms of the size of the effect, it was reasonably large, with students exposed to video performing about 0.28 standard deviations better. To provide some additional context, in my ECONS101 class in A Trimester last year, a 0.28 standard deviation increase in grade would be 6.4 percentage points, or a bit more than one grade point.

In the conclusion, Noetel et al. provide some theoretical foundation for their positive result:

The finding that video was superior to even face-to-face classes may be explained in a few ways. It may be due to the increased ability for students to manage their own cognitive load (e.g., by pausing and rewinding) or because teachers can better optimise cognitive load through editing.

Essentially, giving students more control over their own pace of learning is a good thing, as is editing the video content to focus more on the key points. Noetel et al. also note that videos perform better even when the face-to-face classes had more time:

...because teachers are prioritising relevant content, by editing out discussions and details that are not important for the learning objectives.

I'm not sure that all teachers would see that as a good outcome. Also, it appears that not all video content is created equal. Extending their analysis a bit further, they find that:

...half the implementations of exchanging other learning for video will be helpful (50% of true effects, 95% CI [40%, 59%]). A small proportion of implementations may be unhelpful for student learning (19% of true effects, 95% CI [13%, 25%]) with the rest having negligible influences.

That suggests that there is a large amount of heterogeneity in the impact of video. So, it is worth considering what conditions make video content more effective. On that question, they find that:

...effects were not significantly different between studies that used videos in lectures, tutorials, or homework... In contrast, the comparison condition was a significant moderator... when video replaced static media (e.g., text) it was significantly more effective... than when video replaced a teacher...

So, the comparison really matters here. Replacing the textbook with video led to a 0.51 standard deviation increase in performance, but replacing a teacher (presumably, a face-to-face lecture or tutorial) with video led to a barely statistically significant 0.18 standard deviation increase in performance. The type of content also appears to matter:

Video was more effective when students were assessed on skill acquisition... compared with assessments of their knowledge...

In other words, a 'how-to' style video helping students develop skills was more effective than a video delivering knowledge, skills-development videos increasing performance by 0.44 standard deviations while knowledge videos increased performance by a barely statistically significant 0.18 standard deviations. It may be that videos help students to develop particular skills (Noetel et al. use the example of learning how to calculate a t-statistic), but don't really help in terms of developing a broader knowledge base. That probably also explains two other results from the paper. First:

The number of minutes of the educational intervention did not moderate effects... and there were no significant differences in effects when the video intervention was applied to a single topic or a whole course... In other words, there was no significant dose-response effect.

If one video is good for learning, more videos should be better. However, that is not what they find. Perhaps there are substantial diminishing marginal returns to the use of video in teaching and learning, and all of the benefits of using video are exhausted after the first topic worth of videos is made available to students? That seems unlikely. More likely is that each intervention replaced all the skills-development content with video first (since those are the easiest videos to create), quickly exhausting all of the gains from the transition to video content. However, that doesn't quite explain why there would be no difference between using video in a single topic or a whole course. Definitely, this is something that needs further exploration. Second:

The relative interactivity was a significant moderator of effects... There was no benefit to video when the control condition was afforded more interactivity... Videos were effective when both conditions were given equivalent opportunities for interactivity... Effects were particularly large when videos were presented in an interactive context (e.g., co-viewing with a peer) that was not available to the control condition...

Interactivity really matters. Simply replacing face-to-face classes (or a textbook) with video content that lacks interactive elements does not afford students with the same learning opportunities - the effect of non-interactive video content was small and statistically insignificant. Since interactivity in 'knowledge videos' is more difficult to pull off than in skills-development videos (where students can be encouraged to follow along and practice their skills), that may help explain the type of video content that works best.

Finally, when video doesn't replace the traditional content but is instead added as supplementary material, the effect is large - a 0.88 standard deviation increase in performance, and almost all implementations of adding video increase student performance, which is much less heterogeneity than observed for replacing content with video. These particular results seemed to hold equally for both knowledge and skills-development videos. However, again there appeared to be no dose-response relationship. The takeaway from this is that, at the minimum, once face-to-face teaching returns we should be routinely recording our existing lectures and making those recordings available to students. Teachers need to get over their fear that making recorded lectures available somehow makes students worse off, because it clearly is not the case.

Noetel et al. note in the Discussion section of their paper:

As universities move toward online learning through multimedia, some academics may fear that students will be receiving an inferior learning experience compared with traditional methods. Our review suggests those intuitions are unfounded.

I don't think I would go so far as to say that the intuitions are unfounded. Video clearly adds value in some circumstances, but it is not clear that it dominates face-to-face learning. If the focus of a particular class is on developing particular skills, video is a good tool. Otherwise, unless the teacher has a particular tool or method for engaging interactively with students during their video watching (and I'm still waiting to see robust evaluations of such methods), it isn't clear that there is much advantage over face-to-face teaching. Certainly, this paper doesn't suggest that we should be abandoning face-to-face teaching in favour of video.

Read more:

• Online vs. blended vs. traditional classes
• Flipped classrooms work well for top students in economics
• Flipped classrooms are still best only for the top students
• Online classes lower student grades and completion
• Online classes may also make better students worse off

Trade openness, air pollution, and the pollution haven hypothesis

Until a couple of years ago, in my ECONS102 class (at that time, it would have been ECON110) I used to go through a fairly detailed summary of the evidence in favour of, and against, globalisation. The 'globalisation debate' material covered many dimensions, one of which was the environment. Among the arguments against globalisation in terms of its environmental impacts is the pollution haven hypothesis. When a globalised firm has choices over which country to locate manufacturing operations in, they are likely to choose the location with the lowest levels of environmental protection, because that would entail the lowest cost to the firm. So, if this results in a relocation of manufacturing from high-cost, high-environmental-protection countries to low-cost, low-environmental-protection countries, it effectively exports pollution to the low-environmental-protection countries. Those countries provide a 'haven' for pollution.

Evidence that would support this hypothesis would include countries (or parts of countries) that are more open to trade experiencing higher levels of pollution. So, I think that is why I had this 2017 paper by Faqin Lin (Central University of Finance and Economics, China), published in the journal China Economic Review (sorry, I don't see an ungated version online), waiting on my pile of papers to read. Lin uses Chinese prefecture-level data on exports and imports, and pollution data from NASA (to overcome any data-quality issues related to using Chinese pollution data) over the period from 2004 to 2011. Using distance to the coast as an instrument for trade allows Lin to extract plausibly causal estimates of the impact of trade openness on pollution. They find that:

...the coefficients for trade openness show that a 1% expansion in trade openness quantitatively raises NO2 (Aerosols) concentration by approximately 0.736–1.383% (0.723–0.806%) on average...

In other words, trade openness causes higher levels of pollution in China. The results are robust to alternative data sources (including Chinese pollution data), and different specifications produce similar results. That includes Lin's preferred analysis where they first use distance to the Huai River as an instrument for pollution (because of differences in access to coal-fired heating between the north and south of China) to account for reverse causation, then use the residuals from that analysis as the measure of trade openness. I'm less convinced by this analysis, but the results are at least consistent with the others.

Overall, this research provides some support in favour of the pollution haven hypothesis that differs from the usual cross-country analyses, and therefore doesn't suffer from being confounded by unobserved differences between countries (although you may argue that there are unobserved differences between Chinese prefectures, at least the regulatory system is plausibly consistent).