Shots fired at the end of a debate on contingent valuation

I have written a number of posts about debates on the contingent valuation method (most recently here, but see the links at the end of this post for more). A 2016 debate that I blogged about here, was picked up again in 2020 (but I didn't blog about it then because I was kind of busy trying to manage the COVID lockdown-online teaching debacle). So, what happened? The first of two 2020 articles published in the journal Ecological Economics (sorry I don't see an ungated version online) is by John Whitehead (Appalachian State University), a serial participant in contingent valuation debates.

This part of the debate centres on 'adding up tests', which essentially test for scope problems. To reiterate (from this post):

Scope problems arise when you think about a good that is made up of component parts. If you ask people how much they are willing to pay for Good A and how much they are willing to pay for Good B, the sum of those two WTP values often turns out to be much more than what people would tell you they are willing to pay for Good A and Good B together. This issue is one I encountered early in my research career, in joint work with Ian Bateman and Andreas Tsoumas (ungated earlier version here).

An 'adding up test' tests for whether the willingness to pay for the global good (Good A and Good B together) is more than adding the willingness to pay for Good A alone to the willingness-to-pay for Good B alone. In relation to this particular debate, Whitehead summarises where we are up to:

Desvousges et al. (2012) reinterpret the two-scenario scope test in Chapman et al. (2009) as a three-scenario adding-up test. They then assert that the implicit third willingness-to-pay estimate is not of adequate size. Whitehead (2016) critiques the notion of the adding-up test as an adequacy test and proposes a measure to assess the economic significance of the scope test: scope elasticity. Chapman et al. (2016) argue that Desvousges et al. (2012) misinterpret their scope test. Desvousges et al. (2016) reply that they did not misinterpret the Chapman et al. (2009) scope test and assert that their adding-up test in Desvousges et al. (2015) demonstrates one of their points.

Desvousges et al. (2015) field the Chapman et al. (2009) survey with new sample data collected with a different survey sample mode than that used by Chapman et al. (2009) and three additional scenarios. Desvousges et al. (2015) conduct an adding-up test and argue that willingness-to-pay (WTP) for the whole should be equal to willingness-to-pay for the sum of four parts (the first, second, third and fourth increment scenarios). Desvousges et al. (2015) find that “The sum of the four increments … is about three times as large as the value of the whole” (p. 566).

Whitehead joins the debate on the side of Chapman et al., defending them by examining Desvousges et al.'s analysis and showing that it actually does meet an 'adding up test', thereby showing that there are no scope problems in the original Chapman et al. paper. Whitehead concludes that there are a number of problems in the Desvousges et al. analysis:

First, they do not elicit WTP estimates explicitly consistent with the theory of the adding-up test. Their survey design suggests that a one-tailed test be conducted where the sum of the WTP parts is expected to be greater than the WTP whole. Second, there are several data quality problems: non-monotonicity, flat portions over wide ranges of the bid function and fat tails. Each of these data problems leads to high variability in mean WTP across estimation approach and larger standard errors than those associated with nonparametric estimators that rely on smoothed data.

I'm not going to get into the weeds here, because what I want to highlight is the response by William Desvousges, Kristy Mathews (both independent consultants), and Kenneth Train (University of California - Berkeley), also published in the journal Ecological Economics (and also no ungated version available). The response is only two pages long, and is a very effective takedown of Whitehead. Along the way, Desvousges et al. note that Whitehead:

...made numerous mistakes in his calculations... When these errors are corrected, adding-up fails for each theoretically valid parametric model that Whitehead used.

One example of Whitehead's errors is:

He used medians for the tests instead of means, assuming – incorrectly – that the sum of medians is the median of the sum.

That's a fair criticism. However, Desvousges et al. are not satisfied leaving it at that. Instead, they go onto the attack:

Also, we examined the papers authored or co-authored by Whitehead that are cited in the recent reviews... These papers provide 15 CV datasets. Each of the three problems that Whitehead identified for our paper is evidenced in these datasets:

• Non-monotonicity: 12 of the 15 datasets exhibit non-monotonicity.
• Flat portions of the response curve: All 15 datasets have flat areas for at least half of the possible adjacent prompts, and 4 datasets have flat areas for all adjacent prompts.
• Fat tails: In our data, the yes-share at the highest cost prompt ranged from 15 to 45%, depending on the program increment. In Whitehead's studies, the share ranged from 14 to 53%.

If Whitehead's data are no worse than typical CV studies, then his papers indicate the pervasiveness of these problems in CV studies.

Ouch! That seems to have ended that particular debate. My takeaway (apart from not messing with Desvousges et al.) is that the contingent valuation method is far from perfect. In particular, it is vulnerable to scope problems (which my own research with Ian Bateman and Andreas Tsoumas (ungated earlier version here) showed some years ago. Ironically, that contingent valuation has particular problems is a message that John Whitehead himself has also argued (see here).

Read more:

• Nothing good happens after midnight when using the CVM
• The ongoing contingent valuation debate
• More contingent valuation debates
• Yet another contingent valuation debate

Don't expect to see a Danish Grand Prix any time soon

Big events are fun, and draw in large crowds. But by itself, that doesn't mean that big events are worth the cost. Someone, often but not always taxpayers, has to be willing to pay the cost of hosting. A rational decision-maker would only be willing to host the event if the benefits outweigh the costs. This is a point that I'll be teaching in my ECONS102 class next week, so I was interested to read this recent article by Christian Gjersing Nielsen (Danish Institute for Sports Studies), Søren Bøye Olsen (University of Southern Denmark), and Arne Feddersen (University of Copenhagen), published in the Journal of Sports Economics (sorry, I don't see an ungated version online).

Nielsen et al. focus on the case of a return of the Danish Formula One Grand Prix (which was last held in 1962, although a Danish Grand Prix for Formula Three cars was last run in 1995). They focus on this because:

In 2017, a Danish consortium of private investors presented a plan to host a Formula 1 (F1) Grand Prix... in Copenhagen in 2020, 2021, and 2022.

Ultimately, the plan fell through because it required government funding, and while the national government seemed supportive, but only if the Copenhagen City Council contributed financially. The Council ultimately withdrew its support for the event, and the idea never progressed. Nielsen et al. ask whether the Copenhagen public would actually have been willing to fund the costs of the event, which are significant:

...hosting an F1 Grand Prix in Copenhagen would cost approximately €58 million (adjusted to 2023 prices) annually, including salaries (€15.5 million) and temporary stands (€14 million)... plus an additional annual fee of between €14 million and €50 million for hosting to the rights owners, Liberty Media Corporation...

Nielsen et al. undertook a survey of Copenhagen residents, asking a hypothetical question about their willingness to pay (WTP) for a Grand Prix to be hosted in Copenhagen. Specifically:

Respondents were then asked to imagine that Liberty Media had approved Copenhagen hosting F1 in 2026, 2027, and 2028 and that the private and government funding was already in place. To make hosting conditioned on their response, they were also told that Copenhagen would only accept hosting the race if enough taxpayers would support a temporary municipal tax increase at the household level... Following this, respondents were randomly assigned to two groups. The first group was told that the tax amount that they would have to pay if Copenhagen ended up hosting F1 would depend on their household income... Respondents assigned to the second group did not receive this information and were instead asked to state their household income in the latter part of the survey...

This is an application of the contingent valuation method (which is quite a polarising method, with many debates that I have written about, most recently here). Their sample is about 2000 people, once they exclude 'protest' responses, and just 1452 in their preferred 'weak knife-edge' sample - those who were responsive to a difference in the cost of hosting the event. Based on their range of samples, Nielsen et al. find that:

...mean annual WTP (in the 3 years that Copenhagen hosts F1) estimates between €22.95 and €36.94, while the weighted models result in mean WTP estimates between €24.34 and €43.07, with €30.24... as our central estimate due to the theoretical considerations about consequentiality... Extrapolating our mean WTP estimates to the 320,825 households in Copenhagen Municipality, the aggregated annual WTP (in each of the 3 years that Copenhagen hosts F1) is between €7.36 million and €13.82 million, with €9.70 million (n=1,452, weighted) being our central estimate.

This compares unfavourably with the costs. Specifically:

...the public costs—ignoring indirect or intangible costs—would amount to between €14.4 and €21.6 million annually. Based on our central mean estimate of €9.70 million, the benefits for the households in Copenhagen make up between 44.9% and 67.4% of the public costs, which does not justify hosting F1.

The Copenhagen public are not willing to pay enough to cover the costs of hosting a Grand Prix. So, don't expect to see a Danish Grand Prix any time soon.

Willingness-to-pay for working from home

Jobs come with both monetary and non-monetary characteristics. The monetary characteristics include the salary or wage (obviously) and other monetary benefits. The non-monetary characteristics include how pleasant or unpleasant the job is, how clean or dirty, and how safe or risky. An important non-monetary characteristic of jobs that has become particularly important since the pandemic is the flexibility to work from home. However, it isn't clear whether working from home is a positive or negative characteristic. Many people prefer to work from home, but many others don't. Incidentally, I'm in the latter group, because we only have a small house, and working from home entails working at the dining room table.

Now, non-monetary characteristics of jobs can give rise to wage differences between jobs - what economists refer to as compensating differentials. Jobs with desirable non-monetary characteristics tend to have lower wages than jobs with undesirable monetary characteristics (for an extreme example, see here). One way of explaining this is that many fewer workers want to work in jobs that have undesirable characteristics, and that lower supply of labour leads to higher wages. In other words, workers are essentially compensated for taking on jobs with undesirable non-monetary characteristics.

What does a compensating differential look like for working from home? I wrote about this last year, but the research I referred to there didn't look specifically at compensating differentials. In contrast, this new article by Akshay Vij (University of South Australia) and co-authors, published in the Journal of Economic Behavior and Organization (open access, with non-technical summary on The Conversation), does. They use data from a survey of 1113 employees conducted in Australia in 2020-21, where:

Respondents with an on-site job that had some ability to be done remotely were presented with multiple stated preference experiments where they were offered a choice between job arrangements with different salaries, and differing degrees of flexibility with regards to when and where job tasks needed to be performed... Each respondent was shown 8 scenarios, and the job attributes were varied systematically across scenarios...

This is what economists refer to as a discrete choice experiment, since research participants are asked to make a discrete choice among alternatives, which have different characteristics. Since each research participant makes many such choices, that data can be used to extract the marginal willingness-to-pay for each of the characteristics. In this case, the characteristics included the 'flexibility to work remotely on some days', and the 'flexibility to work remotely at some hours'. So, this research essentially worked out how much workers were willing to pay (that is, how much salary or wage they were willing to give up) in order to have the ability to work remotely.

Vij et al. then used a latent class model to identify four different groups of research participants based on their different responses, as shown in Table 2 from the paper:

Each class represents a roughly equal share of the sample of research participants. However, only two of the groups (Class III and Class IV) value the ability to work remotely some of the time. Vij et al. summarise the results as:

Across our sample, the average worker is willing to forego roughly AUD$3000 - AUD$6000 in annual wages to have the ability to work remotely some workdays and/or workhours. Given that the average respondent in our sample earns roughly $73,000 in annual wages, this implies a compensating wage differential of 4 – 8 per cent. However, median values are lower at AUD$1000 - AUD$1800, or roughly 2 per cent of average annual wage, due to considerable heterogeneity across the four classes. Classes 1 and 2 together comprise 54.3 per cent of the sample population, and do not have a statistically significant preference for either the ability to work remotely some workdays and/or workhours, and therefore have a corresponding wage differential of $0. Class 3 is willing to forego roughly 3 - 5 per cent of average annual wages (AUD$2000 - AUD$4000) to have the ability to work remotely some workdays and/or workhours, and Class 4 is willing to forego 16 – 33 per cent (AUD$12,000 - AUD$24,000) for the same.

Interestingly, the different classes differ on their beliefs about remote work, and Vij et al. note that:

...we observe that Class 1 is less optimistic than the other classes about the quality and quantity of work that can be done remotely, explaining their lower marginal willingness to pay for the ability to work remotely, and Class 4 is most optimistic, explaining their higher marginal willingness to pay...

Next, we compare responses to attitudinal indicators measuring impacts on human relations. Here, Class 2 has significantly greater concerns than the other classes, explaining their lower marginal willingness to pay for the ability to work remotely. In particular, workers belonging to Class 2 are more concerned on average about the negative impacts on their relationships with their colleagues, supervisors and the firm as a whole, as well as opportunities for learning and career advancement.

When you think about the types of jobs that each class predominantly engages in (from Table 2 above), this makes a lot of sense. Class 1 is mostly clerical and administrative workers, but Class 2 is mostly managerial workers, and the latter probably rely more on interpersonal relationships in their work that would be negatively impacts by remote work. In contrast, Class 3 and 4 workers are mostly professional workers, likely to be more self-directed and in some cases fairly autonomous.

However, in light of recent increases in remote work, this was interesting:

Interestingly, in terms of experience with remote working, individuals belonging to Class 2 were more likely to have had greater experience with remote working arrangements prior to the pandemic than other classes...

And yet, those workers had a zero compensating differential for remote work. That is, they didn't value working from home. The survey was conducted in 2020/21, when many of us were experiencing large-scale remote work for the first time. As other workers gain more experience with remote work, I wonder whether the Class 3 and Class 4 workers will be as positively inclined towards remote work in the future. This is something that deserves further investigation.

Finally, in terms of demographics, there was little difference between the classes, although:

...we find that women are most likely to belong to Class 4, and have a significantly higher valuation for remote working. This is consistent with previous studies that have found that women value job flexibility more than men, due to greater caregiving and other responsibilities...

This has interesting implications for the gender wage gap. Since women are more likely to choose flexible work arrangements, and are willing to pay (through lower wages) for the flexibility that remote work provides, should a zero gender wage gap be the appropriate goal, or a zero gap accounting for differences in flexible work arrangements? Again, this is something that deserves further consideration.

Remote work isn't going away any time soon. Some of us might think that the flexibility is a good thing for all workers, but it is clear that not all workers themselves feel that way.

Read more:

• Remote work, wages, and compensating differentials

Student preferences for online vs. in-person education

I've posted a couple of times before about research on student preferences for online or in-person education (see here and here). The takeaway from the two research papers I referenced in those posts was that on average students had a preference for in-person classes, because they were willing to pay less for online options, but a substantial minority of students do prefer online classes. A new article by Lauren Steimle (Georgia Institute of Technology) and co-authors, published in the journal Socio-Economic Planning Sciences (ungated earlier version here), adds a bit more detail to our understanding of these varying student preferences.

Steimle et al. collected survey data in June and July 2020 from 398 Georgia Tech industrial engineering students across all levels of undergraduate study. Importantly, their survey included a discrete choice experiment (DCE), which allows them to evaluate the willingness-to-pay of students for different characteristics of a return to study in the Fall semester of 2020. The characteristics (referred to as attributes) that Steimle et al. investigated:

...were Mode of Course Delivery, Safety on Campus, Residence Hall Operating Capacity, Tuition Reduction, and Limits on Events and Social Gatherings. Each of these attributes was assigned 4 levels based on the latest recommendations from the Centers for Disease Control and Prevention’s interim guidance to institutions of higher education to prepare for COVID-19...

Included the tuition reduction attribute allows Steimle et al. to measure the intensity of students' preferences for the other attributes in terms of how much percentage equivalent tuition reduction they are worth (which they refer to as tuition percentage point equivalent, or TPPE). Before getting to analysing the DCE, Steimle et al. look at the factors associated with students preferring online rather than in-person courses generally, finding that:

i. Students with greater current concern level are more likely to choose online courses than students with lower concern levels,

ii. Students with higher perceived risk of infection are more likely to choose online courses than those with lower perceived risk of infection,

iii. Students with better current living suitability for online courses are more likely to choose online courses than students with worse current living suitability for online courses,

iv. Students who are more risk-seeking are more likely to choose in-person courses than students who are less risk-seeking, and

v. Younger students are more likely to choose in-person courses than older students.

For the DCE, Steimle et al. employ a latent class analysis, which means that the research participants are first sorted into groups (classes), where each class has different preferences than the other classes. In this case, the optimal solution has three classes (although, looking at their diagnostics, I'm a little confused as to why they didn't choose a four-class approach, because it does appear that four classes fits the data better). The three classes differ most in terms of how concerned the students are about the pandemic, labelling the three classes 'low-concern', 'moderate-concern', and 'high-concern', where:

...29% of students fell into Class 1, being not-so-concerned, while 17% fell into Class 3, being highly concerned. Most of the students were in the “moderate-concern” class (i.e., there was an average 54% probability of belonging to Class 2).

Looking at the types of students in each class, Steimle et al. note that:

...students in the “moderate-concern” class or “high-concern” class had slightly better current living suitability for online courses... and were more likely to live off campus in Fall 2020...

Asian/Pacific Islander students were more likely to be in Class 2 or Class 3, while White/Caucasian students were more likely to be in Class 1... Students in the “moderate-concern” class or “high-concern” class tended to politically lean Democrat and had a more liberal world view. “Low-concern” students were inclined to lean Republican and had a more conservative world view. We did not observe substantial differences in the share of those on financial aid among the three classes.

In terms of the DCE results (analysed separately for each class of students), Steimle et al. find that:

...students in more concerned segments (i.e., Class 2 and Class 3) placed more importance on modes incorporating at least some online courses. However, students in Class 1 did not like having online courses compared to completely in-person courses. Students in the “high-concern” segment (Class 3) put much more importance on entirely online courses when deciding to enroll compared to students in the “moderate-concern” segment (Class 2), indicated by the result that the TPPE of “All courses delivered entirely online” in Class 3 was more than twice as large as the corresponding TPPE in Class 2.

Not too surprising there. In general, the high-concern students tend to prefer online classes, while the low-concern students tend to prefer in-person classes. However, that doesn't provide too much help to universities trying to decide what mix of attributes would best ensure high enrolments. Fortunately, Steimle et al. outline some scenarios based on different combinations of attributes:

The first is a “business-as-usual” scenario in which courses are delivered entirely in-person, no requirement on mask-wearing and no testing, 100% operating capacity for residence halls, full tuition, and no limit on the size of social gatherings. Under a “business-as-usual” scenario, the low-concern class (Class 1) was predicted to enroll with 98.9% probability and the “moderate-concern” class (Class 2) was predicted to enroll with 94.8% probability. However, the “high-concern” class (Class 3) was predicted to enroll with only 17.0% probability. Weighting by the class membership probabilities, the weighted average enrollment probability is 82.8% for this “business-as-usual” scenario. In contrast, another tested scenario is a “completely online” scenario in which a 5% tuition reduction is given and has weighted average enrollment probability of 94.6% with the low-concern class enrolling with 85.4% probability, the moderate-concern class enrolling with 99.7% probability, and the high-concern class enrolling with 93.9% probability. A scenario with a higher enrollment probability is a “strict on-campus hybrid” scenario in which large courses are delivered online with small courses delivered in-person, required mask-wearing and extensive testing, residence halls are at 25% capacity (in which there are no roommates and no shared bathrooms), no tuition reduction, and a limit of 20 people at social gatherings. This “strict on-campus hybrid” scenario has a weighted average enrollment probability of 97.6% because it broadly appeals to students from the different classes: low-concern class is expected to enroll with 94.7% probability, the moderate-concern class has a near 100% probability of enrolling, and the high-concern class has a 95.1% probability of enrolling.

All of this seems to accord with the results in the earlier studies, but with a bit more detail in terms of analysis. However, it pays to bear in mind that this study was limited to industrial engineering students, who might have greater preferences for in-person education due to the need for in-person labs. On the other hand, students with greater concern for their health might have been more likely to respond to the survey (although the latent class analysis segregated them into their own analysis, the proportions of students of each type would be biased, and therefore so would the enrolment scenarios that Steimle et al. presented).

Finally, given that the survey was undertaken while there was still great uncertainty over the state of the pandemic and future risk, we probably can't extrapolate from it to understanding student preferences for online or in-person classes outside of pandemic times. For that, we would need to replicate the study at a time when in-person classes are less inherently risky for students.

Read more:

• Columbia University students are willing to pay less for online tuition than for studying in person
• The value of an in-person university education

The willingness to pay for wine bullshit

Consumers often can't tell the difference between two similar substitute products. I've blogged previously about bottled water, but people can't even tell the difference between dog food and pâté (ungated earlier version here). In the bottled water research, people couldn't match different bottled waters to their descriptions. That may be because water descriptions are mostly bullshit - after all, this article by Richard Quandt (Princeton University) notes that wine descriptions are mostly bullshit too.

That brings me to this recent article by Kevin Capehart (California State University), published in the Journal of Wine Economics (sorry, I don't see an ungated version online). Capehart looks at the bullshit wine descriptors that were described as bullshit in Quandt's article (descriptors such as 'silky tannins', 'velvety tannins', 'brawny', and a flavour of 'smoked game'). He then uses various methods to estimate consumers' willingness to pay for bullshit. Specifically, Capehart employs three methods:

I start by using a hedonic regression similar to regressions used by previous studies on wine prices and descriptions...

The second method uses the same dataset used for my hedonic regression, but I draw on approaches for matching rich texts... in order to obtain matching estimates of consumers’ MWTP [marginal willingness-to-pay] for select descriptors. My third method is a stated-preference survey in which I directly ask approximately 500 wine consumers about their MWTP for select descriptors.

For the hedonic regression model, Capehart draws on the descriptions in various online wine catalogues, leading to a dataset of over 51,000 wines. Looking at the effect of different descriptors on wine prices, he finds that:

Despite their joint significance, many of the descriptors have effects that are not statistically significantly different from zero at conventional levels. Examples of descriptors with statistically insignificant effects include “silky” and “silky tannins.”...

Some descriptors do have effects that are statistically different from zero. Of the 106 descriptors, 43 have effects that are significantly significant at the 10% level. Yet, some of those statistically significant effects are not substantively significant. For example, the effect of “velvety tannins” is statistically different from zero (p-value = 0.03), but it is arguably small at only 2.5% (se = 1.1%). A 2.5% change is less than a $1 change for any bottle under $40.

So, some descriptors are associated with higher priced wines, and some with lower price wines. However, mostly the effects are small. Unfortunately, overall the hedonic regression model poses more questions than it answers. As Capehart notes:

If those hedonic results are momentarily accepted, there is much to puzzle over. Why would consumers be willing to pay so much more for wine if an expert described it in terms of the smoked game? How much more or less would they be willing to pay if the expert described the game as being prepared differently, such as by roasting, steaming, or boiling? And what if the expert described the game as a specific type of animal such as a pheasant, boar, deer, squirrel, or some elusive or imaginary creature that few if any have tasted? Questions abound.

Capehart then moves onto using a text-matching estimator:

Any matching estimator tries to match subjects who have received a treatment to control subjects who are as similar as possible, except they did not receive the treatment. After matching, the effect of the treatment on an outcome of interest can be estimated by comparing the outcomes of the matched subjects. Here, the “subjects” are wines, the “treatment” is whether a given Quandt descriptor appears in a wine’s description, and the outcome of interest is the wine’s price.

He essentially uses the 'bag-of-words' approach, which really means noting whether each description contains one or more words (that is, the actual context of their use is ignored). This analysis basically compares wines with very similar descriptions, one of which contains the particular descriptor and one of which does not. In this analysis, Capehart finds that the results are:

...generally consistent or not inconsistent with my hedonic estimates.

Ok, so again wine consumers appear to be willing to pay for some descriptors. Why not ask them about it? That's what the final analysis does, based on a stated preference survey of 469 US wine consumers, conducted online. Essentially, each research participant was asked to choose between two wines, with different prices and descriptions. Based on those hypothetical (stated preference) choices, Capehart finds that:

...most consumers have a zero or near-zero MWTP for velvety rather than silky tannins; that would be consistent with “velvety” and “silky” being synonyms and not inconsistent with my hedonic and matching estimates that suggested at most a small price premium for velvety over silky tannins.

Overall, across the three methods, Capehart concludes that (emphasis is his):

One conclusion is that most consumers seem to have little if any MWTP for wines described by most of the Quandt descriptors. My hedonic approach suggested the majority of the descriptors have a price premium of zero or near-zero. My matching and survey estimates were generally consistent or not inconsistent with my hedonic estimates, at least for the select descriptors considered.

The other conclusion is that some consumers have a non-zero MWTP for wines described by some of the Quandt descriptors. The hedonic approach suggested some descriptors have price premiums (or discounts) that are significant in the statistical sense and arguably significant in the substantive sense. My matching approach suggested the same. And my survey approach suggested some expert and novice wine consumers are willing to pay more than nothing for some descriptors.

In other words, most wine consumers are not willing to pay anything for wine bullshit, but some consumers are. If wine descriptors are mostly bullshit (as Quandt claimed), why would any consumers be willing to pay for wines that have those descriptors? That is the question that Capehart doesn't answer. Perhaps those consumers that are willing to pay a positive amount for a particular descriptor, are willing to do so simply because they feel better for knowing that they are consuming something that has been described as having 'velvety tannins' or the flavour of 'smoked game'? We don't know, so more research will be required in order to uncover the answer to the question of why.

The value of an in-person university education

Last year, I posted about this this research on the willingness-to-pay for studying in person. That research was based on a survey of 46 Columbia University public health students. The researchers asked students how much they would be willing to pay in a straightforward way that is open to substantial bias, whereas a better approach would present students with hypothetical scenarios and derive their willingness-to-pay from their choices between scenarios (using either a contingent valuation approach or a discrete choice experiment). I concluded that post with:

Hopefully, someone else is doing research along those lines.

It turns out there was, and the results are reported in this NBER Working Paper by Esteban Aucejo, Jacob French (both Arizona State University), and Basit Zafar (University of Michigan). Specifically, they surveyed over 1500 students at Arizona State University, asking (among other things) how likely they would be to re-enrol in the Fall 2020 semester, under different conditions and at different costs. Importantly, the conditions included: (1) whether the pandemic continued, or was controlled; (2) whether classes would be in person, or remote; and (3) whether campus life and activities were restricted, or could continue as before. Each survey respondent was presented with six scenarios (with combinations of the conditions) at seven different levels of cost. That allowed Aucejo et al. to extract estimates of the willingness-to-pay (WTP) for in-person classes, and for access to the usual campus life and activities. For the sample as a whole, they found that:

...students are willing to pay $1,043 (approximately 8.1% of average annual cost) to have access to [campus social life]... students are willing to pay $547 more per year in order to have in-person classes (relative to remote classes); this represents 4.2% of average annual cost of attending university, and approximately half the WTP for social activities.

So, there is a small, but statistically significant willingness to pay for campus social activities, and for in-person classes. Students are willing to pay more for social activities than for in-person classes. Some might argue that's because Arizona State University has a reputation as a party school, although it is now recognised for its investment in research. More likely, the higher WTP for social activities reflects the availability of substitutes. Online classes are a viable substitute for in-person classes, but online social activities are not much of a substitute for on-campus social activities.

Things get even more interesting when Aucejo et al. look at heterogeneity across the student sample. This is illustrated in Figure 5 from the paper, which plots the cumulative density functions for WTP for social activities and in-person classes:

Concentrating on the WTP for in-person classes, about one-third of students are willing to pay a negative amount for in-person classes. Those students prefer to study online. There is then about 20 percent of students who have WTP of roughly zero, and about half of students have positive WTP for in-person classes. Those students prefer not to study online. The top half of the distribution (positive WTP) is similar for both in-person classes and social activities. There are far fewer students with negative WTP for social activities than there is for in-person classes.

Aucejo et al. then look at WTP across groups, and find that:

...first-generation students' average WTP for in-person classes is only $204 per year, while second-generation students (that is, students with at least one college-educated parent) have an average WTP of $550... First-generation students also appear less willing to pay for campus social activities (on average, $547 per year versus $1,126 for second-generation)... Similar patterns emerge across a number of socioeconomic divides; for example, nonwhite and non-Honors students appear less willing to pay for in-person instruction and social activities, respectively.

Looking closely at their analysis of WTP for social activities, lower-income and first generation students have lower WTP for social activities, but those effects disappear once Aucejo et al. control for hours worked. Students who work more than 20 hours per week (who also happen to be disproportionately lower-income and first generation students) have lower WTP for social activities. It is likely that reflects a time constraint. In relation to WTP for in-person classes, lower-income students have lower WTP, and that effect is persistent even after controlling for other characteristics (including working). None of the other socio-economic characteristics they test are associated with WTP for in-person classes.

There are a number of things we can take away from this paper, in terms of what it implies about the post-pandemic university teaching environment. First, and most importantly, students are heterogeneous in their preferences (this should not come as a surprise). Some students prefer in-person classes, while other students prefer online classes. A one-size-fits-all approach to university education as we come out of the pandemic is clearly not going to be optimal for all students. However, trying to cater to both groups simultaneously (by providing classes that are at once both in-person and online) is not optimal either. This flexible approach (that many universities are currently adopting) increases lecturer workload. Consequently, it likely reduces the quality of teaching they can offer, both to online and in-person students, compared with teaching in a single mode (either in-person or online). That suggests to me that specialisation is going to out-perform the flexible model. Universities that specialise in in-person teaching will do a better job of it than the flexible university, and will be more attractive to students who prefer in-person learning. Universities that specialise in online teaching will do a better job of it than the flexible university, and will be more attractive to students who prefer online learning. The flexible university is stuck in the middle, not catering adequately to either group of students, despite frantically trying to provide for both.

Second, the value of the social interactions that students have on-campus is large and important. This value is mostly lost in the online model. If universities are committed to a flexible approach (despite the problems I just noted), or are providing an online-only model of education, then finding some way of replicating the social activities for remote students is a must. Universities that can do this well will provide significant value to their students. However, time constraints are binding on social activities for lower-income and first generation students. Finding ways to ensure that these students are able to engage in the important social activities that generate lasting social networks or peers and future colleagues, partners, and collaborators is going to be important, both for in-person and online students.

Overall, I note that more students demonstrated a preference for the in-person teaching model. That fits nicely with my priors. It would be interesting to know whether that result holds across more institutions than just Arizona State University.

[HT: Marginal Revolution, back in March]

Read more:

• Columbia University students are willing to pay less for online tuition than for studying in person

Yet another contingent valuation debate

There is something about contingent valuation as a methodology that seems to generate seemingly endless debates in the research literature (see here and here, for example). The contingent valuation method is a type of non-market valuation - a way of valuing goods (and services) that are not (and cannot be) traded in markets. For example, it can be used to value environmental goods (e.g. how much is improved water quality in a stream worth?) or more general public goods (e.g. how much is another bridge across the Waikato River worth?). Essentially, contingent valuation involves presenting research participants with a series of hypothetical choices in order to determine how much they would be willing to pay for the good or service (it is what we refer to as a stated preference method). For instance, you might use a survey that asks whether people would be willing to pay an additional $5 in council property taxes to improve water quality in a particular stream (or $10, or $50, or $500), and then use that data to work out how much (on average) people are willing to pay for improved water quality.

One of the challenges in contingent valuation is whether the payment is expressed as a single one-off payment (e.g. pay $100 once and the water quality will improve) or as a series of regular (e.g. annual) payments (e.g. pay $25 per year and the water quality will improve). The latest debate I read recently addresses this important question.

First, this 2015 article by Kevin Egan (University of Toledo), Jay Corrigan (Kenyon College), and Daryl Dwyer (University of Toledo), published in the Journal of Environmental Economics and Management (sorry, I don't see an ungated version online). Egan et al. argue that there are three reasons that annual payments should be preferred rather than one-off payments, for three reasons:

First, survey respondents are spared from performing complicated present value calculations. When respondents compare their known annual WTP... to a proposed annual payment, discount rates cancel out in their benefit-cost analysis. When asked to make a one-time payment for a long-lasting environmental improvement, respondents must know their personal discount rate and perform the relevant present value calculation, while having complete fungibility of their income across time and no binding budget or liquidity constraints... Second, we conduct a convergent validity test by comparing CV estimates from surveys with one-time and ongoing annual payments to annual consumer surplus estimates from a travel cost analysis. We demonstrate that CV estimates from surveys with ongoing annual payments better match annual travel cost consumer surplus estimates. Third, a behavioral argument based on mental accounting... suggests that survey respondents who mentally set aside a fixed annual dollar amount for charitable giving will feel more constrained by a large one-time payment compared to a relatively small annual payment.

In other words, annual payments work better because they are easier for respondents to interpret, they are more consistent with revealed preference results based on actual behaviour, and they are consistent with a mental accounting story that notes that a one-off payment has an outsize effect on people's reported willingness-to-pay (WTP), compared with smaller annual payments. They go on to support their results with data from a contingent valuation survey of 967 people, investigating their willingness to pay for water quality improvements at Maumee Bay State Park in Ohio.

Enter the second article, by John Whitehead (Appalachian State University), also published in the Journal of Environmental Economics and Management, but in 2018 (ungated earlier version here). Whitehead attacks the original article on the basis of the convergent validity tests. Specifically, looking at the proportion of survey respondents who were willing to pay at different price levels, he notes that as the price increases a smaller proportion should be willing to pay that amount (because demand curves are downward sloping), but that isn't always the case. Although Egan et al. employ a method that allows them to deal with this issue (by merging parts of the sample where the downward sloping demand curve assumption would be violated), Whitehead shows that an alternative method demonstrates that both one-time payments and annual payments have convergent validity, and he concludes that there is no reason therefore to prefer annual payments.

In the third article, published in the same issue of the Journal of Environmental Economics and Management (ungated earlier version here), Egan et al. respond. They extend their earlier analysis and demonstrate convergent validity using different estimators and a variety of sensitivity analyses. However, in order to demonstrate this validity, they first merge data on two different types of annual payments: a perpetual payment (one that would go on forever) and an annual payment for a fixed period of ten years. 

Finally, Whitehead follows up with a further re-analysis in this article, published in Econ Journal Watch (in 2017, weirdly before the previous two articles appeared officially in print - I guess JEEM had had enough of the debate and declined to publish this rejoinder). He concludes that:

Whenever bids are pooled using dichotomous choice data, the researcher implicitly acknowledges that something went wrong with the execution of the study or with the contingent valuation method itself. It might be that (a) bid subsamples are too low to generate enough power to conduct the statistical test, (b) bids are poorly designed (too close together, too far apart, or there is inadequate coverage of the range of WTP), or (c) contingent valuation method respondents are highly inconsistent... Data cleaning and pooling bids should not be considered a valid research method when the research goal is conducting validity tests over payment schedules or any other issue in the contingent valuation method.

Whitehead's criticism is quite valid. Manipulation of data to enable an analysis may be fine when a lower-bound (or upper-bound) estimate is all that is required. When we are testing the validity of a particular method, the standard of proof is somewhat higher. However, along the way the debate lost sight of two things. First, Egan et al. merged two different annual payment types in their broader analysis. They had earlier demonstrated in the convergent validity tests that perpetual annual payments resulted in similar WTP estimates as the travel cost method. To me, they really showed that perpetual payments should be preferred over a fixed number of annual payments, and over one-off payments. When they merged the two annual payment types together, that nuance was lost. Second, convergent validity was only one of the three reasons that Egan et al. originally proposed for why annual payments should be preferred. The debate focused on only one of the reasons, but the other two remain valid.

Overall, my takeaway is that more follow-up research is definitely required on the convergent validity question, but in the meantime it is likely that we should prefer perpetual annual payments in contingent valuation surveys. Whitehead argues that we should use both a one-time payment and annual payments. Given the potential for survey respondent fatigue and confusion, I wouldn't favour that approach. Contingent valuation solves a real problem - valuing environmental and other goods that are not traded in markets. However, we need to recognise that, like any method, it has its limitations.

Read more:

• Nothing good happens after midnight when using the CVM
• The ongoing contingent valuation debate
• More contingent valuation debates

Columbia University students are willing to pay less for online tuition than for studying in person

Going into lockdown forced university teaching online. We heard a lot about how students were unhappy with online learning (e.g. see here and here). Students felt shortchanged by the new learning environment. So, if students prefer studying in person, we would expect them to be willing to pay for in-person classes compared with online study.

A new working paper by Zafar Zafari (University of Maryland), Lee Goldman, Katia Kovrizhkin, and Peter Muennig (all Columbia University), looks at exactly that question. They surveyed 46 Columbia University public health students, and the study had two interesting parts to it. First, they asked students to trade off the risk of becoming infected by coronavirus against attending classes in person. Second, they asked students how much they were willing to pay for online classes, in comparison with face-to-face classes. They found that:

On average, students were willing to accept a 23% (SE = 4%) risk of infection on campus over the semester in exchange for the opportunity to attend class in-person. Of the 46 students, 37 (80%) were willing to accept a >1% chance of infection and 3 (7%) were willing to accept a 100% chance of infection. One student was not willing to attend classes in-person unless the risk was 0%, and 9 (20%) were willing to attend in-person classes if the risk was less than 1%.

With respect to costs, students were willing-to-pay an average of only 48% (SE: 3%) of their tuition if courses were held exclusively online. No student was willing to pay full price for exclusively on-line instruction, and the maximum reported willingness-to-pay for online-only courses was 85% of standard tuition.

In other words, students in this sample are willing to accept a fairly high risk of coronavirus infection in exchange for attending classes in person, and they're willing to pay much less for online studying (and, by extension, willing to pay much more for the opportunity to attend classes in person). Of course, this should not be the last word on this topic. It was a study of just 46 students, and the methods are not what I would have used.

In fact, I wouldn't read much at all into the willingness-to-pay results - they simply asked students what they were willing to pay, which we know will be biased downwards (people will always say they are willing to pay less than they actually are, if only just in case they are later asked to actually pay!). They also anchored the willingness to pay by giving students a value first, then asking them what they would be willing to pay. It should be a surprise that the average result is about half of what they started with. It seems to me that a student, not knowing how much they would actually be willing to pay but knowing for sure that they wouldn't want to pay the full price, is likely to choose half price. And that's what they did.

A contingent valuation approach or a discrete choice experiment, where student respondents were asked to choose across a range of scenarios incorporating different levels of coronavirus risk, tuition costs, and whether studying was online or in person (and maybe other factors such as class size), would lead to much more plausible and defendable results. Hopefully, someone else is doing research along those lines.

[HT: Marginal Revolution]

Return migrants are willing to accept lower wages in exchange for better institutional quality

Last year, I wrote a post about how return migrants to Vietnam prefer areas with higher-quality institutions. The post was based on the research of one of my PhD students, Ngoc Tran, myself, and Jacques Poot. Ngoc recently submitted her PhD thesis for examination, which is a great achievement. Along the way, she completed four research papers (see here, here, here, and here), and also has a forthcoming book chapter. In this post though, I want to focus on the fourth research paper, as it is the most novel.

In the paper, co-authored between Ngoc Tran, Jacques Poot and I, we looked at the intensity of migrants' preferences to high-quality institutions back in their home country. In other words, we asked how much things like absence of corruption, political stability, and the rule of law, mattered for migrants' decision-making about potentially returning home. To measure the intensity of preferences, we used a novel application of the contingent valuation method.

To do this, we first recognised that there are compensating differentials for working in different locations - in areas with higher-quality amenities (such as higher-quality institutions), wages will be lower than in areas with lower-quality amenities (such as lower-quality institutions). We can exploit this to work out what higher-quality institutions are worth, by looking at how much a migrant's income would have to change to make them indifferent between the area with lower-quality institutions and the area with higher-quality institutions.

We did this by asking Vietnamese migrants in New Zealand two questions:

1. Given your perceptions of the difference in institutional quality between New Zealand and Viet Nam, what would be the smallest level of weekly income before tax in Viet Nam where you would be happy moving back to Viet Nam permanently?; and
2. Now imagine that the institutional quality in Viet Nam changed so that it was equal to New Zealand in all ways (and everything else remained the same). If this happened, what would be the smallest level of weekly income before tax in Viet Nam where you would be happy moving back to Viet Nam permanently?

The first question allowed us to estimate the compensating different based on the current differences in institutional quality and other amenities between the two countries, as well as migration costs. The second question modifies the institutional quality in Vietnam so that it is equal to that in New Zealand, holding everything else (including migration costs) constant. The difference between the answer to Question 1 and the answer to Question 2 provides an estimate of the willingness of the migrant to accept lower wages in exchange for improved institutional quality in Vietnam.

Our estimates show that willingness to pay for an incremental unit improvement in institutional quality in Viet Nam is, on average, NZD 79.80 per week (approximately 33 percent of the average weekly wage in Viet Nam for the same period). Moreover, older migrants are willing to pay more, as are migrants who perceive institutional quality to be more important for the repatriation intentions.

As far as we know, this is the first paper ever to use contingent valuation to measure the intensity of preference for institutional quality, certainly among migrants if not among any population group. Notwithstanding the continuing debate on the use of the contingent valuation method (which I've written about here, here, and here), this was a really innovative piece of work.

Congratulations again to Ngoc on submitting her PhD thesis!

Read more:

• Return migrants to Vietnam prefer areas with higher-quality institutions

More contingent valuation debates

Back in March I highlighted a series of papers debating the validity (or not) of contingent valuation studies. To recap:

One non-market valuation technique that we discuss in ECON110 is the contingent valuation method (CVM) - a survey-based stated preference approach. We call it stated preference because we essentially ask people the maximum amount they would be willing to pay for the good or service (so they state their preference), or the minimum amount they would be willing to accept in exchange for foregoing the good or service. This differs from a revealed preference approach, where you look at the actual behaviour of people to derive their implied willingness-to-pay or willingness-to-accept.

As I've noted before, I've used CVM in a number of my past research projects, including this one on landmine clearance in Thailand (ungated earlier version here), this one on landmines in Cambodia (ungated earlier version here), and a still incomplete paper on estimating demand for a hypothetical HIV vaccine in China (which I presented as a poster at this conference).

One of the issues highlighted in that earlier debate had to do with scope problems:

Scope problems arise when you think about a good that is made up of component parts. If you ask people how much they are willing to pay for Good A and how much they are willing to pay for Good B, the sum of those two WTP values often turns out to be much more than what people would tell you they are willing to pay for Good A and Good B together. This issue is one I encountered early in my research career, in joint work with Ian Bateman and Andreas Tsoumas (ungated earlier version here).

Which brings me to two new papers published in the journal Ecological Economics. But first, let's back up a little bit. Back in 2009, David Chapman (Stratus Consulting, and lately the US Forest Service) and co-authors wrote this report estimating people's willingness-to-pay to clean up Oklahoma’s Illinois River System and Tenkiller Lake. In 2012, William Desvousges and Kristy Mathews (both consultants), and Kenneth Train (University of California, Berkeley) wrote a pretty scathing review of scope tests in contingent valuation studies (published in Ecological Economics, ungated here), and the Chapman et al. report was one that was singled out. You may remember Desvousges, Mathews, and Train from the CVM debate I discussed in my earlier post.

Four years later, Chapman et al. respond to the Desvousges et al. paper (sorry I don't see an ungated version online). In their reply, Chapman et al. demonstrate a quite different interpretation of their methods that on the surface appears to validate their results. Here's their conclusion:

In summary, DMT argue that Chapman et al.'s scope difference is inadequate because it fails to satisfy theoretical tests related to discounting and to diminishing marginal utility and substitution. Also, according to them, our scope difference is too small. Once the fundamental flaws in their interpretation of the scenarios are corrected, none of these arguments hold. The upshot is that Chapman et al. must be assigned to the long list of studies cited by DMT where their tests of adequacy cannot be applied.

However, Desvousges et al. respond (also no ungated version available). The response is only two pages, but it leaves the matter pretty much resolved (I think). The new interpretation of the methods employed by Chapman et al. has raised some serious questions about the overall validity of the study. Here's what Desvousges et al. say:

...this statement indicates that respondents were given insufficient information to evaluate the benefits of the program relative to the bid amount (the cost.) The authors argue that the value of the program depends on how the environmental services changed over time, and yet the survey did not provide this information to the respondent. So the authors violated a fundamental requirement of CV studies, namely, that the program must be described in sufficient detail to allow the respondent to evaluate its benefits relative to costs. The authors have jumped – to put it colloquially – from the frying pan into the fire: the argument that they use to deflect our criticism about inadequate response to scope creates an even larger problem for their study, that respondents were not given the information needed to evaluate the program.

All of which suggests that, when you are drawn into defending the quality of your work, you should be very careful that you don't end up simply digging a bigger hole for your research to be buried in.

The ongoing contingent valuation debate

Economists tend to value goods and services at their market value. There is some intuitive sense to this - the value of a good or service is whatever someone is willing to pay for it. However, not all goods and services can be valued in this way, because not all things of value are able to be traded in markets (e.g. clean air) or are not traded because they don't exist yet (e.g. vaccines for HIV). In these cases, we need to use non-market valuation techniques if we want to derive estimates of value.

One non-market valuation technique that we discuss in ECON110 is the contingent valuation method (CVM) - a survey-based stated preference approach. We call it stated preference because we essentially ask people the maximum amount they would be willing to pay for the good or service (so they state their preference), or the minimum amount they would be willing to accept in exchange for foregoing the good or service. This differs from a revealed preference approach, where you look at the actual behaviour of people to derive their implied willingness-to-pay or willingness-to-accept.

As I've noted before, I've used CVM in a number of my past research projects, including this one on landmine clearance in Thailand (ungated earlier version here), this one on landmines in Cambodia (ungated earlier version here), and a still incomplete paper on estimating demand for a hypothetical HIV vaccine in China (which I presented as a poster at this conference).

The CVM has faced a number of critics over the years. The criticisms are essentially based on whether the estimates provided by the method are fit-for-purpose. That is, does the CVM actually measure the values that it sets out to measure?

The latest contributions to the CVM debate were published in the latest issue of Applied Economic Perspectives and Policy, with the concluding paper titled "Interesting questions worthy of further study: Our reply to Desvousges, Mathews, and Train’s (2015) comment on our thoughts (2013) on Hausman’s (2012) update of Diamond and Hausman’s (1994) critique of contingent valuation". Before I get to that paper though, it's worth me backtracking a bit to earlier in the debate.

Jerry Hausman (MIT) has been one of the staunchest critics of the CVM, and re-sparked the debate with this 2012 article in the Journal of Economic Perspectives (ungated version here). In the article, Hausman reiterates a number of his earlier critiques. Hausman notes there are three long-standing problems with the CVM:

1) hypothetical response bias that leads contingent valuation to overstatements of value; 2) large differences between willingness to pay and willingness to accept; and 3) the embedding problem which encompasses scope problems.

He further argues that:

respondents to contingent valuation surveys are often not responding out of stable or well-defined preferences, but are essentially inventing their answers on the fly, in a way which makes the resulting data useless for serious analysis.

The hypothetical response bias arises because survey respondents are being asked hypothetical questions - usually they are being asked about their willingness-to-pay for goods that are not traded (perhaps through a small increase in taxes), or for goods that do not yet exist. Because these questions are hypothetical, there is little incentive for respondents to answer in a way that is consistent with what they would do if actually faced with the choice.

The differences between willingness to pay (WTP) and willingness to accept (WTA) are common to CVM studies (e.g. my co-authors and I found this difference in a study of landmine clearance in Thailand (ungated earlier version here)). For rational decision-makers the difference between willingness-to-pay to receive a benefit, and willingness-to-accept a payment in exchange for not receiving that same benefit should be the same. But it turns out that WTP is generally lower than WTA. Many argue that this is consistent with quasi-rational decision-making, i.e. loss aversion leads us to be willing to pay less for something we don't have than what we would be willing to accept to give up that same item - an endowment effect.

Scope problems arise when you think about a good that is made up of component parts. If you ask people how much they are willing to pay for Good A and how much they are willing to pay for Good B, the sum of those two WTP values often turns out to be much more than what people would tell you they are willing to pay for Good A and Good B together. This issue is one I encountered early in my research career, in joint work with Ian Bateman and Andreas Tsoumas (ungated earlier version here).

Fast-forward to 2013, and this article in Applied Economic Perspectives and Policy, by Timothy Haab (Ohio State), Matthew Interis and Daniel Petrolia (both Mississippi State), and John Whitehead (Appalachian State). Haab et al. respond to each of Hausman's critiques. In terms of the first critique, they note that current approaches are reducing hypothetical bias using a range of methods, including asking respondents to sign an 'oath' before responding to the survey. In terms of the WTP-WTA difference, they note (as I did above) that the existence of endowment effects makes these differences consistent with behavioural economic theory. And in terms of scope problems, they note that issues of scope are consistent with diminishing marginal utility (the WTP for Good A depends on whether Good B is already provided or not) and substitution between market and non-market goods. Haab et al. conclude:

in direct response to Hausman’s selective interpretation of the literature, we believe that the overwhelming amount of evidence shows: (1) the existence (or nonexistence) of hypothetical bias continues to raise important research questions about the incentives guiding survey responses and preference revelation in real as well as hypothetical settings, and contingent valuation can help answer these questions; (2) the WTP-WTA gap debate is far from settled and raises important research questions about the future design and use of benefit cost analyses in which contingent valuation will undoubtedly play a part; and (3) CVM studies do, in fact, tend to pass a scope test and there is little support for the argument that the adding up test is the definitive test of CVM validity.

And onto the latest contributions to the debate. This paper (sorry I don't see an ungated version) by William Desvousges and Kristy Mathews (both consultants), and Kenneth Train (University of California, Berkeley) responds to Haab et al., argues against a number of specific statements in the Haab et al. paper. They argue that they are highlighting "the limitations of current approaches to guide future research".

Finally, Haab et al. respond (in the paper with the beautifully long title cited above, no ungated version available that I can see), noting that their responses in the earlier piece were based on 'best' practice, not current practice. That is a bit of an indictment of current CVM practice - if 'best' practice is known but not currently followed, then questions would rightly be raised about the reliability of the results of CVM studies.

However, I'm not convinced that in all cases 'best' practice has yet been identified. As Haab et al. note, these issues (especially dealing with hypothetical bias and scope issues) are interesting, and worthy of future study. Especially since at least one of my PhD students will be using CVM in their current research.

Read more:

• Nothing good happens after midnight when using the CVM

Nothing good happens after midnight when using the CVM

Some goods and services that have value to people are not traded in markets (e.g. clean air) or are not traded because they don't exist yet (e.g. vaccines for HIV). These goods and services have value (we like clean air, and we want more of it), but direct estimates of the value cannot be made. However, if we want to evaluate the costs and benefits of different project alternatives, or if we want to evaluate the demand for products that don't exist yet, we need to get some sense of these values.

Economists often use non-market valuation techniques to derive estimates of value for goods and services that are not actively traded in markets. One non-market valuation technique that we discuss in ECON110 is the contingent valuation method (CVM) - a survey-based stated preference approach. We call it stated preference because we essentially ask people the maximum amount they would be willing to pay for the good or service (so they state their preference), or the minimum amount they would be willing to accept in exchange for foregoing the good or service. This differs from a revealed preference approach, where you look at the actual behaviour of people to derive their implied willingness-to-pay or willingness-to-accept.

I've used the CVM in a number of past studies, including this one on landmine clearance in Thailand (ungated earlier version here), this one on landmines in Cambodia (ungated earlier version here), and a still incomplete paper on estimating demand for a hypothetical HIV vaccine in China (which I presented as a poster at this conference). And I have a current PhD student who may use the CVM in evaluating the intensity of preferences for institutional quality by return migrants to Vietnam.

While the CVM is attractive because it is fairly intuitive and easy to use, we know that there are a number of issues with it. Not the least is that people just aren't very good at estimating what they are willing to pay (or accept) for hypothetical goods or services, or in hypothetical scenarios. And people can be quite inconsistent in their responses (which violates the common economic assumption of static preferences, which you may or may not adhere to). Either way, that leads to very noisy measures of value.

A recent paper by David Dickinson and John Whitehead (Appalachian State University) demonstrates the challenge of obtaining 'good' estimates of value quite clearly (sorry I don't see an ungated version anywhere). They use data from an online survey of Appalachian State students, who were asked whether they would vote for a student-funded group to purchase, install, and operate a wind turbine as part of a Renewable Energy Initiative. If the wind turbine was purchased, students would face a fee of $X. The student fee ($X) varied between $4 and $56 in the survey, and students could vote "For" or "Against". The key element of the study though, was that the students were randomised to complete the survey at different times of the day or night. Now, the time of day (or night) that you complete a survey shouldn't in theory affect how you feel about renewable energy, or how much you would be willing to contribute to this initiative. But it turns out it does matter:

During morning, afternoon, and evening time blocks, students vote rationally with "for" votes declining as the student fee rises... During the night time, students are completely insensitive to the student fee, at least in the standard way of thinking; the student fee has no statistically significant effect on "yes" votes.

In other words, the students demonstrated a downward-sloping demand curve during the morning, afternoon, and evening, but price didn't seem to matter at night (between Midnight and 6am). The lack of price sensitivity at night leads the authors to conclude that "Nothing good happens after midnight when using the CVM". I would add that it suggests we should all avoid watching the Shopping Channel late at night.