Durban dodges a bullet by losing the Commonwealth Games

I read with interest this week that Durban has been stripped of the hosting rights for the 2022 Commonwealth Games. The New Zealand Herald reported on Tuesday:

Durban was stripped of the right to host the 2022 Commonwealth Games mainly because the South African government couldn't provide financial guarantees.

Also, other commitments the city made when it won the bid had still not been met nearly two years later.

Durban presented a revised budget and hosting proposal to the Commonwealth Games Federation over the weekend but the last-ditch effort to save Africa's first international multi-sport event wasn't enough.

"It is with disappointment that the detailed review has concluded that there is a significant departure from the undertakings provided in Durban's bid, and as a result a number of key obligations and commitments in areas such as governance, venues, funding and risk management/assurance have not been met," the CGF said in a statement.

The CGF was "actively exploring alternative options, including a potential replacement host," CGF president Louise Martin said.

Those of us who have read Andrew Zimbalist's excellent book "Circus Maximus: The Economic Gamble behind Hosting the Olympics and the World Cup" (which I reviewed here) will know that this outcome is probably a good thing for the government and the people of Durban. There is little evidence that there is any short-term or long-term positive impact on the economy of hosting a large event like the Olympics, Commonwealth Games, or FIFA World Cup.

However, I did read this paper recently, by Paul Dolan (London School of Economics) and others, which shows a short-term increase in subjective wellbeing (happiness) in London at the time of the Olympics, compared with Paris and Berlin. However, the increase in happiness was short-lived, and had disappeared within a year of the event.

So, perhaps the people of Durban will be less happy in 2022 than they would have been while hosting the Commonwealth Games, but by 2023 they will be just as happy having not hosted it. And they won't be saddled with a bunch of white elephant venues and a significant public debt to pay off.

Read more:

• Book review: Circus Maximus
• The economic non-impact of stadiums and arenas

[HT: Tim Harford for the Dolan et al. paper]

Take your pick: Economists are dodgy researchers, or not

I've written a couple of times here about economics education and moral corruption (see here and here). If economists were really corrupting influences though, you would probably expect to see them engage in dodgy research practices like falsifying data. I was recently point to this 2014 paper by Sarah Necker (University of Freiburg) published in the journal Research Policy (sorry I don't see an ungated version anywhere online).

In the paper, Freiburg reports on data collected from 426 members of the European Economic Association. She asked them about the justifiability of different questionable research practices, about whether they had engaged in the practice themselves, and whether others in their department had done so. Here's what she found in terms of views on justifiability:

Economists clearly condemn behavior that misleads the scientific community or causes harm to careers. The least justifiable action is “copying work from others without citing.” Respondents unanimously (CI: 99–100%) agree that this behavior is unjustifiable. Fabricating or correcting data as well as excluding part of the data are rejected by at least 97% (CI: 96–99%). “Using tricks to increase t-values, R2, or other statistics” is rejected by 96% (CI: 94–98%), 93%(CI: 90–95%) consider “incorrectly giving a colleague co-authorship who has not worked on the paper” unjustifiable...

Strategic behavior in the publication process is also rejected but more accepted than practices applicable when analyzing data or writing papers. Citing strategically or maximizing the number of publications by slicing into the smallest publishable unit is rejected by 64% (CI: 60–69%). Complying with suggestions by referees even though one thinks they are wrong is considered unjustifiable by 61% (CI: 56–66%)...

So, these practices are all viewed as unjustifiable by the majority of respondents. Does that translate into behaviour? Freiburg reports:

The correction, fabrication, or partial exclusion of data, incorrect co-authorship, or copying of others’ work is admitted by 1–3.5%. The use of “tricks to increase t-values, R2, or other statistics” is reported by 7%. Having accepted or offered gifts in exchange for (co-)authorship, access to data, or promotion is admitted by 3%. Acceptance or offering of sex or money is reported by 1–2%. One percent admits to the simultaneous sub-mission of manuscripts to journals. About one fifth admits to having refrained from citing others’ work that contradicted the own analysis or to having maximized the number of publications by slicing their work into the smallest publishable unit. Having at least once copied from their own previous work without citing is reported by 24% (CI: 20–28%). Even more admit to questionable practices of data analysis (32–38%), e.g., the “selective presentation of findings so that they confirm one’s argument.” Having complied with suggestions from referees despite having thought that they were wrong is reported by 39% (CI: 34–44%). Even 59% (CI: 55–64%) report that they have at least once cited strategically to increase the prospect of publishing their work.

According to their responses, 6.3% of the participants have never engaged in a practice rejected by at least a majority of peers.

You might think those rates are high, or low, depending on your priors. However, Freiburg notes that they are similar to those reported in a similar study of psychologists (see here for an ungated version of that work). Other results are noted as being similar to those for management or business scholars.

Freiburg then goes on to demonstrate that these questionable behaviours are related to respondents' perceptions of the pressure to publish. That is, that those facing greater publication pressure are more likely to engage in questionable research behaviours. Those results are not nearly as clear, as they are for the most part statistically insignificant, mostly likely due to the relatively small sample size. So, although they provide a plausible narrative, I don't find them convincing.

However, the takeaway message here clearly depends on your own biases. Either economists are dodgy researchers, frequently engaging in questionable research practices ("only 6.3% have never engaged in a practice rejected by at least a majority of peers"), or they are no better or worse than other disciplines in this regard. Take your pick.

[HT: Bill Cochrane]

The irrationality of NFL play callers, part 2

A few weeks ago I wrote a post about the irrationality of NFL offensive play callers, specifically that they fail to adequately randomise their play choices, with the implication that defensive play callers should be able to (and do) exploit this for their own gain. Why would they do this? The Emara et al. paper (one of the two I used in the post) suggested:

Perhaps teams feel pressure not to repeat the play type on offense, in order to avoid criticism for being too “predictable” by fans, media, or executives who have difficulty detecting whether outcomes of a sequence are statistically independent. Further, perhaps this concern is sufficiently important so that teams accept the negative consequences that arise from the risk that the defense can detect a pattern in their mixing.

Which seems like a plausible suggestion. Last week I read this recent post by Jesse Galef on the same topic:

In football, it pays to be unpredictable (although the “wrong way touchdown” might be taking it a bit far.) If the other team picks up on an unintended pattern in your play calling, they can take advantage of it and adjust their strategy to counter yours. Coaches and their staff of coordinators are paid millions of dollars to call plays that maximize their team’s talent and exploit their opponent’s weaknesses.

That’s why it surprised Brian Burke, formerly of AdvancedNFLAnalytics.com (and now hired by ESPN) to see a peculiar trend: football teams seem to rush a remarkably high percent on 2nd and 10 compared to 2nd and 9 or 11.

What’s causing that?

Galef argues that there are two possibilities (note that the first one is similar to the suggestion by Emara et al.):

1. Coaches (like all humans) are bad at generating random sequences, and have a tendency to alternate too much when they’re trying to be genuinely random. Since 2nd and 10 is most likely the result of a 1st down pass, alternating would produce a high percent of 2nd down rushes.

2. Coaches are suffering from the ‘small sample fallacy’ and ‘recency bias’, overreacting to the result of the previous play. Since 2nd and 10 not only likely follows a pass, but a failed pass, coaches have an impulse to try the alternative without realizing they’re being predictable.

Galef then goes through some fairly pointy-headed methodological stuff, before arriving at his conclusion:

If their teams don’t get very far on 1st down, coaches are inclined to change their play call on 2nd down. But as a team gains more yards on 1st down, coaches are less and less inclined to switch. If the team got six yards, coaches rush about 57% of the time on 2nd down regardless of whether they ran or passed last play. And it actually reverses if you go beyond that – if the team gained more than six yards on 1st down, coaches have a tendency to repeat whatever just succeeded.

It sure looks like coaches are reacting to the previous play in a predictable Win-Stay Lose-Shift pattern...

All signs point to the recency bias being the primary culprit.

However, I'd still like to see some consideration of risk aversion here. Galef controlled for game situation and a bunch of other game- and team-level variables, but not individual-level variables related to the coaches (he did control for quarterback accuracy, but as far as I can see that might be a team-level variable if the team has changed quarterback mid-season).

This is yet more evidence that there is an exploitable trend in NFL offensive play calling, but the reason underlying this trend is still not fully established. Defensive play callers need not care about the reasons why though - they should be adjusting their strategies now.

Read more:

• The irrationality of NFL play-callers

Grade inflation is harming students' learning

I just finished reading a 2010 paper by Philip Babcock (University of California, Santa Barbara) published in Economic Inquiry (ungated earlier version here), which looked at the real costs of grade inflation. In the paper, Babcock makes a convincing argument that grade inflation exhibits a trade-off that ultimately harms students. The trade-off is that, in classes where grades are inflated, students spend less time studying. On the surface, that might seem like the least startling research conclusion ever, but actually it takes some unpicking.

Babcock uses data on course evaluations for "6,753 classes from 338,414 students taught by 1,568 instructors across all departments, offered in 12 quarters between Fall 2003 and Spring 2007" at the University of California, San Diego. Importantly, the dataset included data on the number of hours (per week) that students reported studying in each of their courses, and their expected grade. The average expected grade for each course as a whole is a measured of how the course is graded (he didn't have data on actual grades, but student decision-making is really about perceptions and expectations, so the expected grade data are the right data to use). The simplest analysis is just to graph the data - here is a plot of study time against the average expected grade:

Note the downward trend line, which nicely illustrates that students in classes with higher average expected grades study for fewer hours per week, on average. Babcock notes that this is a general result across all departments. For instance, here is the same graph for economics:

However, he doesn't stop there, applying regression models to control for characteristics of the course, but also instructor-specific effects, course-specific effects and any time trends. So, the regression results can be thought of as representing the answer to the question: if the same instructor, teaching the same course, had higher expected grades, what would be the effect on study time? Here is what he finds:

Holding fixed instructor and course, classes in which students expect higher grades are found to be classes in which students study significantly less. Results indicate that average study time would be about 50% lower in a class in which the average expected grade was an "A" than in the same course taught by the same instructor in which students expected a "C".

Obviously, that's quite a significant effect. And as I see it, it has two negative implications. First, students who study less will learn less. It really is that simple. Second, as Babcock notes in his paper this isn't so much a story of 'grade inflation' as 'grade compression'. Since the top grade is fixed as an A, if an A is easier to get, that compresses the distribution of grades. The problem here is that the signalling value of grades for employers becomes less valuable. If employers can no longer use grades to tell the top students from the not-quite-but-nearly-top students, then the value of the signal for top students is reduced (I've written about signalling in education before here). If grade compression continues, the problem gets even worse.

Of course, the incentives for teachers are all wrong, and Babcock demonstrates this as well. Students give better teaching evaluations to teachers when those teachers are teaching a course where the average expected grade is higher. So, if good teaching evaluations are valuable to the teacher (for promotion, tenure, salary increments, etc.) then there is incentive for them to inflate grades to make students happy and more likely to rate the teaching highly. On a related note, Babcock also finds that:

...even though lower grades are associated with large-enrollment courses, when the same course is taught by a more lenient instructor, significantly more students enrol.

My only gripe with the paper is pretty minor, and is something that Babcock himself addressed (albeit briefly, in a footnote), and that is the lack of consideration of general equilibrium effects. I've had a number of conversations with other lecturers (and students) about making additional resources available for students (past exam papers, worked examples, extra readings, this blog, etc.). The idea is we put in this additional effort in order to help our students to pass our courses. However, the kicker is that if we as teachers put more effort in, then students might re-direct their own effort away from our courses and towards other courses where the effort requirement from them to achieve their desired grades is higher. I have only anecdotal evidence (from talking with students over a number of years) that this unintended consequence occurs, but it is worrying.

In the case of grade inflation, making it easier to get an A in our course may make it easier for the students to do better in other courses, since they re-direct efforts to the other courses they perceive as being more difficult. While it might make the impact of grade inflation higher (since average grades would also increase in courses where grades are not inflated), I don't think you could argue that this is a good outcome at all.

Overall, while this is based on a single study in a single institution, it is reasonably convincing (or maybe that is just confirmation bias?). Grade inflation is not good for students, even if it might be good for teachers.