Some weak evidence in favour of an information intervention in economics to close the gender gap

I've written a couple of times about information interventions designed to attract more female students to study economics (see here and here). The results have generally been disappointing. That shouldn't be too much of a surprise. If it was really simple to get people to change their behaviour with information, then advertising would be far more effective than it actually is.

The latest paper I read on this topic, though, does seem to provide some evidence in favour of a particular information intervention, although the evidence is not particularly strong. This 2019 article by Amanda Bayer, Syon Bhanot (both Swarthmore College), and Fernando Lozano (Pomona College), published in the journal AEA Papers and Proceedings (ungated earlier version here), looks at an intervention applied to 2710 incoming first-year students across nine highly selective liberal arts colleges in 2016. The intervention was targeted at female students and under-represented minorities (URM). As Bayer et al. explain:

We randomly assign all incoming women and URM students to one of three experimental conditions: (i) a control condition with no email messaging; (ii) a “Welcome” treatment that consisted of two emails encouraging students to consider enrolling in economics courses; and (iii) a “Welcome+Info” treatment of two emails that encouraged students to consider enrolling in economics courses, but also included information showcasing the diversity of research and researchers within economics, with links to educational materials on the AEA’s website.

The good news is that the 'Welcome+Info' treatment appears to work, when looking at whether students take an economics course in their first semester at college, as:

...the Welcome treatment is associated with a 1.5 percentage point increase in the probability of taking a course, but this difference is not statistically significant (p = 0.38). However, the Welcome+Info treatment increases the likelihood of completing an economics course by 3.0 percentage points; the effect is statistically significant at the 10 percent level (p = 0.09) and substantial relative to the 15.2 percent probability that a student in the control group takes an economics course...

The effect is relatively large, but only marginally statistically significant. But when Bayer et al. look instead at whether students took any economics course across their whole first year of college, the effect is "positive but smaller and not statistically significant". That suggests that female and URM students may be simply shifting economics courses that they were planning on taking in their second semester into their first semester instead. However, Bayer et al. report that looking at the second semester shows no significant negative effects (which is what you would see if students were transferring from the second semester to the first). Perhaps then it is that their analysis is simply insufficiently statistically powered to detect an effect? The point estimates on the analysis of the full first year imply about a 1.2 percent increase in probability of taking any economics course in the first year of study (and no difference between the 'Welcome' and 'Welcome+Info' treatments). That's not zero, but not particularly large.

However, Bayer et al. also report that:

...the results are particularly striking for first-generation college students, where the Welcome+Info treatment is associated with a 11.4 percentage point increase in the likelihood of taking economics (p = 0.07), while the Welcome email has essentially no effect on these same students.

It wouldn't surprise me that an information intervention has a larger effect on first-in-family students, because those students can't as easily draw on the experience of their family in choosing the best course of study. However, overall this study can be added to the growing collection of weak evidence for the effect of information interventions on students' (and especially female and URM students') choice about whether to study economics or not.

Read more:

• Could role models close the gender gap in economics?
• Could closing the gender gap in economics be as simple as providing students with information?
• Disappointing results on simple information interventions to close the gender gap in economics

This week in research #73

Here's what caught my eye in research over the past week:

• Panos and Wright (open access) examine the gender gap in financial understanding amongst 11-18 year-olds in Scotland, and find that most of the gender gap is explained by male students being more likely to study economics, finance, or business studies
• Ahsan, Panza, and Song (with ungated earlier version here) find that a one-standard-deviation increase in Atlantic trade resulted in a 0.157-standard-deviation decrease in intra-European conflict onset, using data covering the period from 1640 to 1896
• Baselgia and Martınez (open access) find that when some Swiss cantons repealed a controversial tax policy that taxed wealthy foreigners based on their living expenses, their stock of super-rich foreigners dropped by 43% as a consequence (no surprises, the wealthy are quite mobile)
• Einav, Klopack, and Mahoney (with ungated earlier version here) look at consumers who continue to pay for subscriptions they no longer value, and show that cancellation frictions (such as switching costs) roughly double seller revenues on average, holding fixed initial subscribers
• Nicolli, Gilli, and Vona (open access) augment the standard Environmental Kuznets Curve equation with the interaction between income per capita and the Gini coefficient, and find that reducing inequality is beneficial for the environment, especially for rich countries

Clements and Si on what Australian economics PhD students do

Like the article I blogged about on Tuesday, this 2019 article by Kenneth Clements (University of Western Australia) and the late Jiawen Si, published in the journal Australian Economic Review, had been sitting in my to-be-read pile for far too long until I read it last week. Clements and Si provide a description of what Australian economics PhDs do, based mostly on the experience of UWA doctoral students. Although dated, I thought this article provided some interesting insights, such as:

Until recently, that PhD thesis was of the nature of a research monograph, but increasingly is a collection of three core chapters that deal with different aspects of a problem. These chapters are more or less self-contained and are structured such that they could be submitted to a journal with only modest editing.

In New Zealand, most universities now offer a PhD with publication (using the terminology of my institution), and in economics, almost all PhD students go through that pathway. Along with most of my colleagues, I won't consider accepting a PhD student who simply wants to write a single monograph (although that was the form of my own PhD thesis). As Clements and Si note:

The change facilitates subsequent publication of the chapters and has the advantage of dividing up a large study into smaller, more manageable elements that might be easier to complete through greater focus of students’ (and supervisors’) attention. A possible disadvantage of this model is that students no longer gain experience in producing a book-length monograph, but this is probably of minor consequence in the current environment in which the value of books is usually considerably less than the equivalent number of articles.

In my view, student's academic CV is much stronger if it includes a number of high-quality publications, rather than a single thesis monograph. And since most PhD students are aiming to become academics after graduation, the monograph seems like a substantially inferior option.

One aspect of the PhD programmes that surprised me was how many Australian universities require a coursework component of the PhD. Clements and Si look across 13 Australian universities (including all of the Group of Eight), and note that:

...five universities—the Australian National University (ANU), Melbourne, Monash, the University of NSW (UNSW) and the University of Queensland—now have a compulsory coursework component of their PhD requirements. Presumably, this is designed to provide guidance and tools for the subsequent thesis research and to produce graduates whose broader training in economics might benefit their subsequent careers.

It's also likely that those five universities (all of which are in the Group of Eight) have coursework requirements because it more closely mimics the US format of PhD studies, which many more of their academic staff will likely have experienced. As far as I am aware, no New Zealand universities have a coursework requirement as part of their PhD programme, although there may be optional papers that PhD students can take. And of course, many PhD students will 'audit' postgraduate papers, such as in econometrics, in order to develop their research skills.

Finally, Clements and Si look at the outputs from the PhD, focusing on UWA students, and find that:

On average, two publications come out of a thesis... However, if those who publish nothing are excluded, about 3.5 publications emerge from the average thesis. Using the Australian Business Deans Council ranking, roughly 30 per cent of these publications are in highly ranked journals, that is, those ranked A or A*...

That seems not too dissimilar from our students, although I have only ever had one PhD student who completed and published more than three articles directly from their PhD thesis. The high ranking of those publications at UWA probably reflects the high quality of supervision available there.

Anyway, it is interesting to see what Australian economics PhD students do, albeit as of 2019, and to recognise that it is not dissimilar from what happens for New Zealand economics PhDs.

Walstad and Bosshardt on undergraduate GPAs in economics (and other subjects)

I've had this short 2019 article by William Walstad (University of Nebraska-Lincoln) and William Bosshardt (Florida Atlantic University), published in the journal American Economic Review Papers and Proceedings (sorry, I don't see an ungated version online), sitting in my 'to-be-read' pile for far too long (especially given how short it is!). Walstad and Bosshardt (incidentally, two of the top researchers in economics education) look at how GPAs differ across undergraduate majors, using data from the Baccalaureate and Beyond (B&B) project of the National Center for Education Statistics in the US. Their sample covers nearly 16,000 students who graduated in the 2007-08 academic year.

The results make for fascinating reading (albeit, as a snapshot of GPAs that is now over 15 years old). For starters:

The overall undergraduate GPA for all majors is 3.24, or between a B and B+ letter grade. The GPA for economics majors is only slightly below the average at 3.16.

...we also calculated an economics GPA for college graduates who completed a course or courses in economics. This economics GPA average is 2.9, or a B to B− grade.

It's not too surprising to learn that economics has a slightly lower GPA than other subjects, or that students who take an economics course (or more than one), but don't major in economics have a lower GPA in economics than students taking an economics major. I'm sure that the results would be similar for other subjects (with students taking a few courses in a subject having a lower GPA in that subject than students majoring in that subject).

Walstad and Bosshardt then look at the factors associated with GPA:

The most striking finding is that prior achievement or measured ability in high school is highly associated with success in the undergraduate coursework...

The only other variable that appears to be a fairly consistent predictor of GPAs is age. The age effect is nonlinear, with the youngest college graduates having the highest GPA, but it declines with age and then eventually increases.

The correlation of GPA with prior academic achievement is not surprising. Students who do well at high school tend to do better in university as well, on average. The better performing students tend to be highly engaged and motivated, both at high school and university. However, the effect of age is more interesting. The youngest students (those aged under 22 years at graduation) have the highest GPA (of 3.36), and GPA declines with age (to 3.19 for those aged 22 or 23, and 3.02 for those aged 24 or 25), until the oldest group (those aged 26 years or older at graduation), where GPA jumps back up (to 3.29). It is likely that this reflects that students who take longer to get to graduation have lower grades, having failed one or more courses along the way. However, the oldest group will include many 'mature' students, who tend to be more focused on their studies and do better on average. The other variable that stands out as associated with GPAs is gender, with female students receiving a GPA that is 0.14 points higher, on average.

Next, Walstad and Bosshardt look at the factors associated with GPA by subject. Focusing on economics, the factors that are statistically significantly associated with GPA in economics are being aged 24 or 25 at graduation (which is associated with a GPA that is 0.22 points lower, on average), having a high school GPA of 3.5 or more (which is associated with a GPA that is 0.24 points higher, on average), verbal and math SAT scores (which are both associated with higher GPAs), and graduating from a baccalaureate or Masters granting institution, rather than a doctoral degree granting institution. The latter is consistent across all subjects, which suggests that grades are simpler lower on average at doctoral degree granting institutions.

Walstad and Bosshardt, though, focus on the differences by gender, noting that:

Females earned significantly higher overall GPAs than males and in four subjects (biology, calculus, foreign languages, and psychology), but no significant difference is evident in three subjects (economics, business, engineering).

However, female students are less likely than male students to earn an A grade in their first economics course (and their first engineering course), which is not the case for any of the other subjects. Male students in economics get an A grade 3.1 percentage points more often than female students, so the effect is not large.

Given the known issues with grade inflation over time (see here and here), it would be interesting to know how things have changed since 2007-08, and especially whether the gender gap in economics achievement is still apparent. The B&B project does apparently have data for a cohort that graduated in 2015-16, so perhaps a follow-up project is forthcoming?