The gender gap in reviewing and editing for top economics journals

I've written a number of posts about the gender gap in economics (most recently, this one; see the list at the end of this post for more). So, I was interested to read this article by David Card, Stefano DellaVigna (both University of California, Berkeley), Patricia Funk (Universita Della Svizzera Italiana), and Nagore Iriberri (University of the Basque Country), published in the Quarterly Journal of Economics (ungated earlier version here). In the paper, they look at the reviewing and editing process for four top economics journals (Journal of the European Economics Association, Quarterly Journal of Economics, Review of Economics and Statistics, and Review of Economic Studies), in terms of gender bias.

They have data on nearly 30,000 submissions to those four journals, which they use to:

...analyze gender differences in how papers are assigned to referees, how they are reviewed, and how editors use referee inputs to reach a revise and resubmit (R&R) verdict.

There is both good and bad in their results. First, they find that:

...female-authored papers receive 22 log points (std. err. = 0.05) more citations than male-authored papers, controlling for the referee evaluations. Our estimate of this gender gap is robust to alternative measures of citations and to a variety of alternative specifications...

What this means is that:

...female-authored papers would have to be of 28 log points (32%) higher quality than male-authored papers to receive the same referee assessment.

That gender gap is then perpetuated through the publication process:

 On average editors tend to follow the referees’ recommendations, putting essentially no weight on author gender in their R&R decisions. This means that they are overrejecting female-authored papers relative to a citation-maximizing benchmark. 

There are at least a couple of interpretations for what is going on here:

There are two main explanations for our finding that female-authored papers receive more citations, conditional on the referee evaluations. The first is that referees hold female authors to a higher bar, perhaps because of stereotype biases. The second is that female-authored papers have characteristics that lead to higher citations but are not as highly rewarded in the review process. For example, female authors may tend to write more empirically oriented papers, or concentrate on certain topics within broad field categories that referees undervalue relative to expected citations.

Card et al. find evidence that is suggestive that paper characteristics play some role. That is, female authors do concentrate on different fields than male authors, and those fields tend to attract more citations. The news isn't all bad though:

We find no gender differences in the time that referees take to return a recommendation, in the time that editors take to reach a decision, or in the time between submission and acceptance for published papers.

There's also no difference between female and male reviewers, so if male reviewers are holding female-authored papers to a higher bar, then so are female reviewers. However, they couldn't assess differences between female and male editors, because of a lack of female editors (!).

This paper also made me wonder whether there is a third explanation, which the authors did not identify. Female authors might self-censor, sending only their very best papers to these top journals, but sending their second-tier papers to other, lower-ranked journals. In contrast, male authors might send those second-tier papers to the top journals anyway. That would lead the average quality (as measured by citations) of female-authored papers to be higher than male-authored papers.

Regardless of explanation, the results have clear and negative implications for female economists, who have to work harder to get the same outcomes as male economists. And the solutions are not as straightforward as equalising gender participation. Card et al. note in their conclusion that:

One potential remedy to help female economists — using more female referees — is unlikely to help, given that female referees hold female-authored papers to the same higher bar as do male referees.

Their preferred solution is disappointingly vague (although admittedly, I'm not sure I can offer anything better):

It appears to us that a simpler path is to increase the awareness of the higher bar for female-authored papers. The referees and editors can then take it into account in their recommendations and decisions.

Of course, that means moving away from the double-blind reviewing process (which is a fiction in any case - but that's a topic for another post).

Read more:

• A sobering report on the culture in the economics profession
• Could closing the gender gap in economics be as simple as providing students with information?
• Why we should care about the gender gap in economics
• Justin Wolfers on why we should care about the gender gap in economics
• Female representation in economics
• CORE on 'missing women in economics'
• More on the gender gap in economics
• The gender gap in economics