Tuesday, 7 March 2017

Grade inflation is harming students' learning

I just finished reading a 2010 paper by Philip Babcock (University of California, Santa Barbara) published in Economic Inquiry (ungated earlier version here), which looked at the real costs of grade inflation. In the paper, Babcock makes a convincing argument that grade inflation exhibits a trade-off that ultimately harms students. The trade-off is that, in classes where grades are inflated, students spend less time studying. On the surface, that might seem like the least startling research conclusion ever, but actually it takes some unpicking.

Babcock uses data on course evaluations for "6,753 classes from 338,414 students taught by 1,568 instructors across all departments, offered in 12 quarters between Fall 2003 and Spring 2007" at the University of California, San Diego. Importantly, the dataset included data on the number of hours (per week) that students reported studying in each of their courses, and their expected grade. The average expected grade for each course as a whole is a measured of how the course is graded (he didn't have data on actual grades, but student decision-making is really about perceptions and expectations, so the expected grade data are the right data to use). The simplest analysis is just to graph the data - here is a plot of study time against the average expected grade:

Note the downward trend line, which nicely illustrates that students in classes with higher average expected grades study for fewer hours per week, on average. Babcock notes that this is a general result across all departments. For instance, here is the same graph for economics:

However, he doesn't stop there, applying regression models to control for characteristics of the course, but also instructor-specific effects, course-specific effects and any time trends. So, the regression results can be thought of as representing the answer to the question: if the same instructor, teaching the same course, had higher expected grades, what would be the effect on study time? Here is what he finds:
Holding fixed instructor and course, classes in which students expect higher grades are found to be classes in which students study significantly less. Results indicate that average study time would be about 50% lower in a class in which the average expected grade was an "A" than in the same course taught by the same instructor in which students expected a "C".
Obviously, that's quite a significant effect. And as I see it, it has two negative implications. First, students who study less will learn less. It really is that simple. Second, as Babcock notes in his paper this isn't so much a story of 'grade inflation' as 'grade compression'. Since the top grade is fixed as an A, if an A is easier to get, that compresses the distribution of grades. The problem here is that the signalling value of grades for employers becomes less valuable. If employers can no longer use grades to tell the top students from the not-quite-but-nearly-top students, then the value of the signal for top students is reduced (I've written about signalling in education before here). If grade compression continues, the problem gets even worse.

Of course, the incentives for teachers are all wrong, and Babcock demonstrates this as well. Students give better teaching evaluations to teachers when those teachers are teaching a course where the average expected grade is higher. So, if good teaching evaluations are valuable to the teacher (for promotion, tenure, salary increments, etc.) then there is incentive for them to inflate grades to make students happy and more likely to rate the teaching highly. On a related note, Babcock also finds that:
...even though lower grades are associated with large-enrollment courses, when the same course is taught by a more lenient instructor, significantly more students enrol.
My only gripe with the paper is pretty minor, and is something that Babcock himself addressed (albeit briefly, in a footnote), and that is the lack of consideration of general equilibrium effects. I've had a number of conversations with other lecturers (and students) about making additional resources available for students (past exam papers, worked examples, extra readings, this blog, etc.). The idea is we put in this additional effort in order to help our students to pass our courses. However, the kicker is that if we as teachers put more effort in, then students might re-direct their own effort away from our courses and towards other courses where the effort requirement from them to achieve their desired grades is higher. I have only anecdotal evidence (from talking with students over a number of years) that this unintended consequence occurs, but it is worrying.

In the case of grade inflation, making it easier to get an A in our course may make it easier for the students to do better in other courses, since they re-direct efforts to the other courses they perceive as being more difficult. While it might make the impact of grade inflation higher (since average grades would also increase in courses where grades are not inflated), I don't think you could argue that this is a good outcome at all.

Overall, while this is based on a single study in a single institution, it is reasonably convincing (or maybe that is just confirmation bias?). Grade inflation is not good for students, even if it might be good for teachers.

Monday, 6 March 2017

Cocaine makes a comeback in the U.S.

The Washington Post reports:
While much of the recent attention on drug abuse in the United States has focused on the heroin and opioid epidemic, cocaine has also been making a comeback. It appears to be a case of supply driving demand.
After years of falling output, the size of Colombia’s illegal coca crop has exploded since 2013, and the boom is starting to appear on U.S. streets...
Given that we are covering supply and demand in ECON100 at the moment, this seems like an appropriate time to look at what is happening here. There has been an increase in the supply of cocaine (a shift to the right, or down, of the supply curve for cocaine), as shown in the diagram below, from S0 to S1. This lowers the equilibrium price of cocaine (from P0 to P1), and increases the quantity sold and consumed (from Q0 to Q1). In other words, since cocaine is now cheaper, more people use it (rather than more expensive substitutes such as heroin or opioids).

Why has supply increased? The Washington Post article explains:
The State Department report cites four major reasons for the sudden coca-growing binge by Colombian farmers.
The first is that FARC rebels appear to have encouraged farmers in areas under their control to plant as much coca as possible in preparation for the end of the war, “purportedly motivated by the belief that the Colombian government’s post-peace accord investment and subsidies will focus on regions with the greatest quantities of coca,” the report said.
At the same time, the government reduced eradication in those areas “to lower the risk of armed conflict” and create a favorable climate for the final peace settlement.
The Colombian government also ended aerial spraying with herbicides in favor of manual eradication. But when eradication brigades have arrived to tear out illegal crops, local farmers have blocked roads and found other ways to thwart the removal, including the placement of improvised explosive devices among the coca bushes.
The final factor is the Colombian government’s financial squeeze, “resulting in a 90 percent reduction in the number of manual eradicators in 2016 as compared to 2008,” according to the report.
Notice how all of the four factors listed above either lead to a greater supply of the crop getting to market, or lower the costs of supplying cocaine. Both of which leads to an increase in supply. However, it's likely that the change in price shown in the diagram above is somewhat exaggerated, in the reverse of the effect explained by Tom Wainwright in his excellent book, Narconomics (see my post here for details).

Read more:


Sunday, 5 March 2017

Beware bogus taxi surcharges when travelling on an expense account

In my ECON100 and ECON110 classes, we discuss moral hazard. In ECON110, we also discuss supplier-induced demand. So, I was interested to read this new paper (ungated earlier version here) by Loukas Balafoutas (University of Innsbruck), Rudolf Kerschbamer (University of Innsbruck), and Matthias Sutter (European University Institute), published in the journal The Economic Record.

In the paper, Balafoutas et al. look at moral hazard among taxi drivers. Recall that moral hazard is the tendency for a person who is imperfectly monitored to engage in dishonest or otherwise undesirable behaviour. That is, moral hazard is a problem of post-contractual opportunism, where people have the incentive to change their behaviour to take advantage of the terms of the agreement (or contract) to extract additional benefits for themselves, at the expense of the other party.

However, what the authors are looking at in this paper isn't your run-of-the-mill moral hazard. They distinguish it as what they call second-degree moral hazard:
As an illustration consider the market for health care services and assume that the consumer of the service – in this case, a patient – is fully insured and interacts with a seller of the service – in this case, a physician. Moral hazard implies that the patient may have incentives to demand more of the service than required (by asking for more numerous or more extensive tests or treatments), since he will not bear its costs. However, the behaviour of the physician may also be affected by the extent of the coverage: if the physician expects that the patient is not concerned about minimising costs, he may be more inclined to suggest or prescribe more expensive treatments. Notice that the two stories – which we will call first-degree moral hazard and second-degree moral hazard – are observationally equivalent in terms of final outcomes, in the sense that more extensive insurance coverage leads to higher expenditure, but the mechanisms are different. While first-degree moral hazard operates through the demand side, second-degree moral hazard increases expenditure through supplier-induced demand – the artificial increase in demand induced by the actions of the seller.
Supplier-induced demand occurs when there is asymmetric information about the necessity for services. In a city that is unfamiliar to the taxi passenger, they may not know the shortest route to get to their destination, whereas the taxi driver does. So, the taxi driver has an incentive to take the passenger on a longer (and more expensive route). Balafoutas et al. note:
In the case of taxi rides in an unknown city, the service traded on the market is a credence good... meaning that an expert seller possesses superior information about the needs of the consumer. In particular, the driver knows the correct route to a destination while the consumer does not. This property of credence goods opens the door to different types of fraud: overtreatment occurs when the consumer receives more extensive treatment than what is necessary to meet his needs (with taxi rides, this amounts to a time-consuming detour); in the opposite case of undertreatment, the service provided is not enough to satisfy the consumer (i.e. he does not reach his destination); finally, in credence goods markets where the consumer is unable to observe the quality she has received, there might also be an overcharging incentive, meaning that the price charged by the seller is too high, given the service that has been provided.
The paper uses an interesting field experiment to tease out how (and by how much) taxi drivers engage in second-degree moral hazard behaviour. They used four research assistants (two male, two female), who each took 100 trips across Athens. All four of them went on each trip within a few minutes of each other (to ensure they faced similar traffic situations, etc.), and two of them (one male, one female) made it clear to the driver that their (the passenger's) employer was paying for the ride (which I guess was true - the researchers no doubt paid for these rides). The research assistants then recorded the price paid for the journey, plus measured the distance travelled using a portable GPS. This allowed the authors to compare rides between male and female passengers, as well as between the 'moral hazard treatment' and control. They found that:
our moral hazard manipulation has an economically pronounced and statistically significant positive effect on the likelihood and the amount of overcharging, with passengers in that treatment being about 17% more likely to pay higher-than-justified prices for a given ride. This also leads to significantly higher consumer expenditures in this treatment on average. At the same time, the rate of overtreatment (by taking time-consuming detours) does not differ across treatments. Hence, second-degree moral hazard does not increase the extent of overtreatment compared to the control, while it does increase the likelihood and the extent of overcharging.
The actual mechanism of the overcharging was interesting:
In the large majority of these cases (86 out of 112, or 76.8%) bogus surcharges were applied, namely higher-than-justified extras from and to the airport, the port, the railway station and the bus station. The second most frequent source of overcharging (14 cases) were manipulated taximeters or the use of the night tariff during daytime, while rounding-up (not tipping!) of the price (by more than 5%) accounted for the remaining twelve cases of overcharging.
I also found it interesting that in the control, female passengers were more likely to pay a higher fare than male passengers. However, in the moral hazard treatment (i.e. when passengers told the drivers their employer was paying) there was no gender difference in overcharging.

Overall, this is a really interesting use of a field experiment to tease out results that would not be easily possible in other ways. The only part of the paper that made me a little concerned was buried in a footnote:
We note that the sample initially consisted of 256 observations collected in 2013. The remaining 144 observations were collected in 2014 at the advice of the editor and referees and resulted in stronger statistical significance. Based on the initial sample alone, some of the results outlined in subsection 2.2 were only marginally significant.
This trick of boosting the sample size in order to generate effects that are statistically significant is a trick that experimental psychologists use, and shouldn't really be condoned. It's been implicated in the replication crisis that psychology is facing. At least they were up-front about it, and the results were published. But imagine, if the results became less statistically significant, then what do they do? Not publish? Or continue to collect additional data, until eventually they attain some statistical significance?

Anyway, that little gripe aside, I found this paper a good read, with important implications. One of which is that I will certainly be looking out for bogus taxi surcharges the next time I take a taxi from the airport in a strange city.

[HT: The Economist]

Saturday, 4 March 2017

Book Review: Economix

Earlier this week I finished reading Economix: How Our Economy Works (and Doesn't Work) in Words and Pictures by Michael Goodwin. This book wasn't really what I expected, but that isn't necessarily a bad thing. I've previously read Grady Klein and Yoram Bauman's The Cartoon Introduction to Economics (both Volume I and Volume II), and was expecting a similar treatment of economic principles, illustrated with some mildly humorous cartoons. Instead, Goodwin takes the reader on a journey through the history of economic thought, from Jean-Baptiste Colbert and mercantilism in France in the 17th Century, through to Occupy Wall Street. The book is ably illustrated by Dan Burr.

One of my criteria for a good pop-economics book is whether or not there is anything in them that I can use in my ECON100 or ECON110 classes, and there were certainly a few bits that will be useful in that sense. I also found the discussion of the physiocrats in France useful for myself, along with the explanation of the centrality of the labour theory of value to Marxism, which I hadn't realised before. I guess that just reflects that I haven't deeply studies the history of economic thought myself.

The general reader will probably find this book interesting, but it is probably a bit America-centric for my tastes. To be fair, Goodwin makes this point early:
...while I tried to cover the whole world, I focused on the economy of the United States because I'm an American and that's the economy I live in.
Fair enough. Some may find the end of the book a little preachy, and if you have strong neoliberal beliefs you'll probably not appreciate many parts of the book. I found it to be a fair treatment, and there is nothing terribly controversial in it apart from perhaps one bit where Goodwin advocates a progressive revenue tax on corporations (which he argues would incentivise the corporations to split into smaller entities, limiting monopoly power).

If you're interested in the history of economic though, but not sure where to start, this is probably a better place than most. There is also a good amount of additional content on Goodwin's website, economixcomix.com. I especially like this timeline of economists.