Three papers on the gender gap in economics seminars

The culture of the economics profession has attracted a lot of attention of late (see this post, or the list of posts at the end of this one for more on the gender gap in economics more generally). Through all of this attention, there have been some interesting and important changes underway (see this post, for example). A reasonable question, then, is whether things are improving.

Three recent papers may provide some early indications, in relation to economics seminars. The first is this NBER Working Paper by Pascaline Dupas (Stanford University) and co-authors. They systematically collected data from 460 economics seminars and job market talks across 32 top universities from January to May of 2019, as well as presentations at the NBER Summer Institute. So, these data don't tell us much about changes over time, but may provide a useful baseline against which to compare progress. Importantly, their data focuses on the questions and interruptions that speakers face, which is a key aspect of the climate of seminars and presentations (see this post, for example). Overall, Dupas et al. find from the seminar and job market talks data that:

On average, roughly 26 questions are asked during a regular economics seminar and 35 questions are asked during a job market talk. For a 90-minute seminar, this represents one interruption every 3.5 and 2.5 minutes, respectively - although interruptions are not uniformly distributed during the time allotted. Moreover, there is considerable heterogeneity with the number of questions ranging from a low of 5 to a high of 69 for any given seminar. There are 3.6 times as many questions from men as from women during regular seminars - and 7.6 times during job market talks - despite men only outnumbering women roughly 2 to 1 in attendance (and 3 to 1 in job talks)...

In terms of the type of questions, roughly 35 percent of all questions in regular seminars (37 percent in job market talks) are classified as clarifications, followed by another 17 percent (13 percent) that are classified as comments. Suggestions, follow-ups, and criticisms each account for 10 percent or less in both regular seminars and job market talks - perhaps countering the reputation that economics is as an overly critical discipline.

Comparing male and female presenters, they find that:

...women presenters are asked 3.8 additional questions (p<0.01) relative to men (a 12 percent increase). Accounting for the influence of a range of other factors about the audience, the presenter, the topic, and the coders, reduces the differential to 2.4 questions (p<0.05). This disparity appears most pronounced during recruitment (“job market”) talks (3.7 extra questions, p<0.05) and regular seminars with an external (rather than in-house) speaker (2.6 extra questions, p<0.05)...

Although we find that women receive a greater number of suggestions and clarifying questions, we also find that they are more likely to be asked questions that are rated as patronizing or hostile.

The latter finding should be particularly of concern. Also:

Aggregating across negative tones (questions which are patronizing, disruptive, demeaning or hostile), women receive 0.5 more such questions, three quarters of which seem to be coming from men.

Dupas et al. also analyse a much more limited dataset from the NBER Summer Institute, but find broadly similar results. However, this dataset allows them to investigate whether the format of the seminars makes a difference. On that point, they find that:

...having a discussant and/or Q&A at the end does not mitigate the differential treatment of women presenters. Indeed, women receive more questions than men even in those presentations that had formal discussants... The only mitigating factor appears to be the “moratorium” on questions in the first 5 or 10 minutes of the talk: with the caveat that this represents a very small sample of presentations (N=45), we find that the moratorium completely undoes (if anything, reverses) the gender gap. And this appears to be the result of fewer “clarifying” questions that end up being deferred anyway or followed up on later when asked too early.

So, there is some suggestive evidence that the format of the seminar may make a difference. In my experience, people interrupting the speaker to ask clarifying questions are often not adding value to the talk for anyone but themselves. The takeaway from this paper, though, should be that economics still has a lot of work to do to improve the seminar climate.

The second paper is this article by Jennifer Doleac (Texas A&M University), Erin Hengel (University of Liverpool), and Elizabeth Pancotti (Employ America), published in the papers and proceedings issue of the American Economic Review (ungated version here). Doleac et al. report on the characteristics of invited seminar speakers in a panel of 66 economics departments (60 from the US, and 6 from outside the US) from 2014 to 2019. So, here we get a longitudinal dimension, although Doleac et al. don't have data on the dynamics within each seminar. They focus on the trends over time, distinguishing between genders and between under-represented minority (URM) speakers and non-URM speakers. Here's the key trends, from their Figure 1:

If you squint your eyes, you may pick up a slight upward trend in the proportion of speakers who are non-URM women, and a slight downward trend for non-URM men (the medians are less noisy, so may better represent the trends over time). However, there has not been much change for URM speakers (men or women). Indeed:

Forty-three of the 66 departments in our sample did not invite a single URM woman to speak during this period; 39 did not invite a single URM man to speak.

Maybe there's some small sign of improvement for non-URM women in terms of their proportion of invited speaking opportunities. However, Doleac et al.'s data only goes up to 2019, and it would be interesting to see how more recent trends have played out, especially given the pandemic. Fortunately, that's pretty much what the third paper does, which is this discussion paper by Marcus Biermann (UC Louvain). Biermann looks at how the coronavirus pandemic has affected economics seminars, using data from the seminar series of 270 institutions worldwide (including 243 universities, 14 central banks, 11 research institutes, and 2 international organisations). The dataset includes over 12,000 seminars over the period from 2018 to 2020. Overall:

At the seminar level, 21.8 percent of the seminars are held by female speakers. The average speaker has about 12.2 years of experience after PhD award. The top 1 percent of researchers in terms of their overall output and in terms of their publication record in the last 10 years account for 7 and 12.5 percent of seminars, respectively. The 200 top young economists held 3.3 percent of the seminars...

Looking at the impact of the pandemic, Biermann finds:

...a 7.47 percentage point increase in the relative likelihood that the seminar speaker after the technology shock is female, which is about 34.3 percent in terms of the pre-technology shock mean.

That is quite a substantial effect. Also, he finds that the increase in likelihood of a speaker being female is larger for distances between the home and host institutions of between 1475 and 5000 kilometres. Biermann suggests that:

This implies that parts of the increase in the share of female speakers are driven by a supply side response for medium length distances. The requirement to travel to medium length distant places and to stay overnight may have hindered women to accept seminar invitations before the technology shock.

 Biermann's paper also shows some other interesting trends separate from the change in gender composition of seminar speakers, including:

...that the overall number of seminars declined and that the decline was not driven by the short-run supply of speakers... The distribution of seminars speakers shifted toward researchers of better quality... The geography of knowledge dissemination changed significantly as the average distance between host and speakers’ institutions increased by 32 percent and the share of seminars across borders also increased. Finally... the inequality in presentation opportunities manifested itself in inequality in citations.

Overall, these three papers provide a lot of food for thought on the state of and trends in economics seminars and the gender gap. Clearly, the seminar climate needs to improve from where it was in 2019. The trends over time suggest only small change in the gender distribution of seminar speakers over time (and it is possible that the difficult climate for female seminar speakers may contribute to that), but the recent shift to online seminar series may be a factor in reducing the gender gap in seminar speakers. However, these three papers also leave a lot of questions unanswered, including whether the change in gender composition of speakers as a result of the pandemic has been sustained as most seminar series moved to online or hybrid formats, whether the change in gender composition was matched by a change in composition in favour of URM speakers, and importantly, if the seminar climate is a problem, how it can best be improved. Hopefully, the increasing research attention in this area will provide us with some answers to these questions soon.

[HT: Marginal Revolution for the Dupas et al. paper, and David McKenzie at the Development Impact Blog for the Biermann paper]

Read more:

• The gender gap in economics
• More on the gender gap in economics
• CORE on 'missing women in economics'
• Female representation in economics
• Justin Wolfers on why we should care about the gender gap in economics
• Why we should care about the gender gap in economics
• Could closing the gender gap in economics be as simple as providing students with information?
• A sobering report on the culture in the economics profession
• The gender gap in reviewing and editing for top economics journals
• The gender gap in U.S. economics education
• UWE and the gender gap in economics (results pending?)