Do men and women pitch science proposals differently, and does it matter for funding outcomes?

If male academics and female academics write academic papers and grant proposals differently, does that lead to different outcomes by gender? Past studies have worried about whether grant funding decisions are affected by gender bias (see here, for example), and differences in writing style may contribute to that. However, the article I discussed in this post from earlier this year concluded that there was little evidence of bias in grant funding, at least since 2000 in the US.

Nevertheless, I thought it would be interesting to read this 2020 article by Julian Kolev (Southern Methodist University), Yuly Fuentes-Medel, and Fiona Murray (both MIT), published in the journal AEA Papers and Proceedings (ungated here), because it not only looks at the grant funding decisions, but also at writing style. Kolev et al. focus on grant applications submitted to the Bill and Melinda Gates Foundation and the National Institutes of Health (NIH) over the period from 2008 to 2016. The sample includes 6931 Gates Foundation applications and 12,589 NIH applications.

Kolev et al. first subject the applications to textual analysis to the abstract of each application, evaluating the positivity of the text (and the extent to which the word "novel" is used), the readability (using the Flesch reading ease score), concreteness of the language (as opposed to abstractness), and three measures of how narrow or broad the abstract is. In this textual analysis, they find that:

...female applicants are less likely to present their research using positive vocabulary, they are more likely to write with high readability, and they prefer concrete language. Moving to our final three measures, we find an interesting dichotomy: even as female applicants use fewer broad words and more narrow words in their abstracts, we find that their research is characterized by lower MeSH concentrations, meaning that they cover a wider range of medical subjects in their work, at least within the NIH sample. Effect sizes are relatively small: the impact of gender ranges from approximately 0.04 to 0.08 standard deviations for our significant effects.

So, there are small but statistically significant differences in writing style between male academics and female academics in these funding proposal abstracts. Does that translate into differences in outcome? Kolev et al. test for whether the measures of writing style correlate with funding outcome, while controlling for:

...calendar time and application topic fixed effects, controls for total word count and the count of relevant words for dictionary-based metrics, and applicant publication history and gender.

In this analysis, Kolev et al. find that:

For Gates applicants, high levels of concreteness tend to improve the odds of funding; by contrast, at the NIH, we find a strong positive impact for MeSH concentration and marginal effects for both broad and narrow words.

So, the evidence is weak that writing style matters, or that writing style differences between the genders affect the success of funding applications. However, not so fast. There is a key problem with this analysis. If you read the quote above about the control variables in this second analysis, you may note that they control for gender. That might sound sensible, but if you're wanting to evaluate whether writing style differences between the genders affect funding outcomes, you don't want to control for both writing style and gender. What Kolev et al. have actually tested is whether writing style differences within each gender affect funding application outcomes, finding that they don't. As an example, their analysis doesn't answer the question of whether readability differences between male and female academics affect funding outcomes, it answers the question of whether readability differences matter overall (which it appears they don't), controlling for the average difference in funding outcomes between men and women. Those are quite different questions.

In other words, we probably want to know whether style mediates the effect of gender on funding outcomes, but this analysis doesn't do that. Instead, they should either run the analysis with gender as the main explanatory variable, then add the style variables and see if the coefficient on the gender variable shrinks, or run the analysis with interactions between gender and the style variables.

The results are, on the one hand, surprising. Quality of writing should matter. Stylistic differences should matter less. However, the quality of the proposed research should matter even more than the quality or style of writing. And this study wasn't even evaluating the quality or style of all of the writing (or the quality of the proposed research), only the style of writing in the abstract for the proposal. That, along with the issue with the second analysis above, make this paper of limited use for understanding whether there is a gender difference in funding outcomes (and, if there is, whether writing style differences contribute to the difference). The difference in writing style is an interesting result in itself, but we need to know more.

Read more:

• The gender gap in economics
• More on the gender gap in economics
• CORE on 'missing women in economics'
• Female representation in economics
• Justin Wolfers on why we should care about the gender gap in economics
• Why we should care about the gender gap in economics
• Could closing the gender gap in economics be as simple as providing students with information?
• A sobering report on the culture in the economics profession
• The gender gap in reviewing and editing for top economics journals
• The gender gap in U.S. economics education
• UWE and the gender gap in economics (results pending?)
• Three papers on the gender gap in economics seminars
• Disappointing results on simple information interventions to close the gender gap in economics
• Gender differences in fields of specialisation within economics
• Gendered citations of papers in the 'top five' economics journals
• The positive effect of mentoring for female economists
• What machine learning is telling us about the gender dynamics in economics seminars
• There's something about Wellesley
• The gender gaps in academia may not arise entirely from gender biases

What the persuasiveness of economics experts and signalling tell us about the gender gap in economics

Are the opinions of expert economists persuasive? The answer to that question will no doubt depend on who you ask. And at least some would answer 'yes, but I wish they weren't'. Does the gender of the economist matter for how persuasive their opinion is? That is a much more difficult question to answer. However, this recent article by Hans Sievertsen and Sarah Smith (both University of Bristol), published in the Journal of Economic Behavior and Organization (open access, but see this paywalled FT article as well), provides us with a starting point. Sievertsen and Smith:

...use an information provision experiment... to test whether the opinion about a topical policy issue expressed by a senior female economist is more, or less, persuasive than the same opinion expressed by a senior male economist. We run the same experiment twice – the first time, members of the public are shown credentials of expertise, the second time they are not.

Specifically, Sievertsen and Smith survey people in the US, asking their opinion on a range of issues (rated on a five-point scale from 'strongly disagree' to 'strongly agree'). Alongside a statement of the issue, each research participant was provided with the opinion of an economist, drawn from the US Economic Experts Panel run out of the University of Chicago. By comparing the results from the survey with those of an earlier survey of people who didn't get to see an economist's opinion, the persuasiveness of the economist whose opinion is provided (on each issue) can be evaluated.

The ten issues are based on the following statements:

1. Use of artificial intelligence over the next ten years will lead to a substantial increase in the growth rates of real per capita income in the US and Western Europe over the subsequent two decades.

2. There needs to be more government regulation around Twitter’s content moderation and personal data protection.

3. It would serve the US economy well to make it unlawful for companies with revenues over $1 billion to offer goods or services for sale at an excessive price during an exceptional market shock. (Price Gouging)

4. Efforts to achieve the goal of reaching net-zero emissions of greenhouse gases by 2050 will be a major drag on global economic growth.

5. Given the centrality of semiconductors to the manufacturing of many products, securing reliable supplies should be a key strategic objective of national policy.

6. A significant factor behind today’s higher US inflation is dominant corporations in uncompetitive markets taking advantage of their market power to raise prices. (Greedflation)

7. Financial regulators in the US and Europe lack the tools and authority to deter runs on banks by uninsured depositors.

8. When economic policy-makers are unable to commit credibly in advance to a specific decision rule, they will often follow a poor policy trajectory.

9. A windfall tax on the profits of large oil companies ‚ with the revenue rebated to households‚ would provide an efficient means to protect the average US household.

10. A ban on advertising junk foods (those that are high in sugar, salt, and fat) would be an effective policy to reduce child obesity.

The economists' actual views on each of the statements is not important for the research question (although the article does tell you, and you can probably guess for some of them what the average opinion of the economists is). Sievertsen and Smith first evaluate how persuasive economists are in general, finding that:

On average, a one-point change on the Likert scale in expert opinion is associated with a 0.17 point change in public opinion... Expert opinions have no effect on public opinions about Greedflation, while there are stronger effects for Price Gouging, Financial Regulation and Economic Policy. There is some support for the argument... that persuaders are more effective when receivers are less certain: The degree of persuasiveness is weaker on issues where baseline public opinion is more certain... The degree of persuasiveness is also stronger on issues where there is less distance between sub-panel expert opinion and baseline public opinion... this suggests that experts may be perceived as less credible when their views are further out of line with those of the general public.

So, the opinions of economic experts are convincing (somewhat). The last point is interesting though - people are most convinced when the economist's views are similar to those of the general public. This suggests some confirmation bias - people believe the experts more when the experts agree with them! It is also interesting who is persuaded most:

The degree of persuasiveness is greater for men [p = 0.002] and for non-whites [p = 0.000]. It is also greater for those with a degree [p = 0.000] and for those with higher self-reported economics knowledge [p = 0.000]. Those who identify as Republicans are also more persuaded by economists’ opinions than Democrats/Independents [p = 0.030].

Those who are more educated, and who claim to have more economics knowledge, are more persuaded by expert economists. More educated people likely give more credence to the views of other educated people, while those who claim to know more economics are more likely to modify their views to fit with those of economics experts.

What about gender differences in persuasion? Sievertsen and Smith switch the analysis to evaluating whether the opinion of the research participants exactly matches that of the expert whose opinion they are provided, and find that:

...members of the public are 1.1 percentage points more likely to match with the opinion of a female expert than with the same opinion expressed by a male economist.

Sievertsen and Smith find similar results when evaluating the distance between the opinion of the research participant and the expert (the distance is smaller for female experts). The effect (1.1 percentage points, against a match rate for male experts of 33.5 percent) seems quite small, but Sievertsen and Smith note that some matches would happen purely by chance, and after accounting for that:

...the degree of persuasiveness of female expert opinions is around 20 per cent higher {than male experts]...

Why are female economics experts so persuasive? Here's where things get interesting. Sievertsen and Smith run their survey a second time. The first time around, research participants were told the name and institutional affiliation of the expert economist (as well as shown their profile photo). In the second survey, research participants were only told the name of the expert economist (and shown the photo). In that second survey:

The overall effect of

female expert on the probability of matching opinion drops from 0.011 in the main experiment to 0.0002 in the follow-up.

The extra persuasiveness of female experts (over and above male experts) disappears! Sievertsen and Smith conclude that:

...in the first experiment, credentials provided an information signal that favoured senior female experts. Removing that signal in the follow-up experiment eliminates the gender difference.

It is worth explaining that last result in a bit more detail, because what it really shows is that the general public recognises the gender bias in top economics institutions.

The quality of a purported economics expert is private information. it is known to the expert themselves, but not known to the public. This is a case of asymmetric information. The expert is the informed party, and the general public is the uninformed party. Since the general public cannot tell high-quality and low-quality experts apart, they might assume that all experts are low quality. How can a high-quality expert instead convince the general public of their high quality? They must credibly reveal their quality to the public - this is called signalling.

For a signal to be effective, it must meet two conditions: (1) it must be costly; and (2) it must be costly in such a way that those with low quality attributes would not want to attempt the signal. Getting a tenured position is signal of high quality for experts. It is costly to get such a position, and it is costly in a way that low-quality experts wouldn't want to attempt it (because they would be unsuccessful in getting tenure anyway). So, a position at a top university is a signal of quality.

Now, why would this signal be even more effective for female economists? If getting a tenured position at a top institution is even more costly for female economists than for male economists, then the quality of the signal is higher for women than for men. And therefore, the general public would be even more believing of the signal for female economists than for male economists. The gender gap in economics is pervasive (see this post, and the posts linked at the bottom of that post). What is interesting is that this gender gap is so well established that the general public is acting on it!

Sievertsen and Smith finish by pointing out a bit of a paradox:

...if senior female economists have greater credibility in the eyes of the public, then why are they less confident in giving their opinion. This remains an open question.

Indeed. Hopefully, at the margin, this research will convince senior female economists to use their persuasiveness for the good of all (where 'good' is defined as persuaded more people to believe in economists' opinions).

The effect of the Dobbs US Supreme Court decision on medical school applications

In 2022, the US Supreme Court ruling in Dobbs v. Jackson Women's Health Organization effectively overturned the famous 1973 Roe v Wade decision, returning the decision about abortion policy to individual states. Many states had laws that would ban abortion if Roe v Wade was ever overturned, and those laws immediately went into effect. Other states responded by enacting legislation to protect the availability of abortion. Overall, the landscape for medical care in the US changed overnight, with different effects in different parts of the country.

How did that change affect the incentives to become a doctor? That is effectively the question addressed in this 2024 article by Joshua Hess (University of South Carolina), published in the journal Economics Letters (ungated earlier version here). Hess looks at the effect of the decision on medical school applications, focusing on the change between years before 2022 (before the Dobbs decision) and 2022 and 2023 (after the Dobbs decision), for states with and without abortion restrictions after the Dobbs decision. In other words, Hess applies a difference-in-differences analysis (evaluating the difference between states with and without abortion restrictions on their books, between the time before and the time after the Dobbs decision, when those restrictions would become effective).

Looking at the share of women applying to medical schools, Hess finds that:

Women’s share of applications to treated schools increased by 0.65% (p-value = 4.8×10−06) relative to control schools in 2022 and by 1.17% (p-value = 3.2 × 10−06) in 2023.

So, Hess finds that women make up a larger share of medical school applications in treated states. I find that a little surprising. But then things get a little weirder, because:

...there is little change in total enrollments over the time frame. It increased from 21,507 in 2018 to 22,845 in 2023. Consequently, changes in women’s (men’s) share strongly imply a similar change in the number of women (men) medical students.

In other words, taking those two results together, states that have abortion bans must have gained more medical school applications from women and fewer applications from men, after the Dobbs decision (and relative to states without abortion bans).

It is difficult for me to understand the decision-making of medical school applicants here. However, I can offer some idle speculation. In states with abortion bans, the status of health care as a profession declined after the Dobbs decision. Moreover, the status of women's health care declined even more, making women's health less attractive as a specialty, compared with other specialties. There is some weak supporting evidence for this, as Hess finds that:

...the share of women applicants to top OB/GYN schools in states with abortion bans increased by nearly 1% more than not top OB/GYN schools in states with abortion bans...

OB/GYN is a medical specialty that attracts more women than men, and the larger effect there than overall is consistent with reducing the status of women's health care. However, if the decrease in the status of health care as a profession was explaining these results, then the total number of medical school applications should decline, when Hess notes that it hasn't changed. Instead, more women are being attracted to medical school, and fewer men.

Are the outside options (the next best alternative to medical school) better for men than for women? I don't know the answer to that question, but even if that were true it would only explain the decrease in male medical school applicants, and not the corresponding increase in female applicants.

That leaves me with one last possible explanation: that women responded to the threat to women's health in states with abortion bans by deciding to become doctors. That would be an effective way of fighting to protect women's health, a form of resistance. So, even as men become less likely to apply to medical school due to the reduction in status of medicine (especially in women's health care), women respond by applying to medical school in greater numbers.

Hess isn't able to extricate the reasons underlying the results he finds. So, my takeaway from this paper is that it raises more questions than it answers. Clearly, this is not the final word on this topic, and hopefully there is other research going on that will help us to better understand why there has been such a gender shift in medical school applications.

Husbands vs. wives

On the Development Impact blog back in March, Markus Goldstein pointed to this fascinating NBER Working Paper by John Conlon (Harvard University) and co-authors, innocuously titled "Learning in the Household". Despite the title, the working paper reveals a study of how husbands and wives treat information revealed by each other differently (a finding that will no doubt come as no surprise to my wife).

Conlon et al. recruited 400 married couples and 500 unrelated strangers (with equal numbers of men and women) for their study, in Chennai (India) in 2019. In the experiment, research participants:

...play five rounds - with different treatments - of a balls-and-urns task... The goal in each round is to guess the number of red balls in an urn containing 20 red and white balls. Participants are informed that the number of red balls is drawn uniformly from 4 to 16 in each round...

In each round, participants receive independent signals about the composition of the urn. Concretely, they privately draw balls from the urn with replacement. Depending on the round, they either play the game entirely on their own or else can learn some of the signals from their teammate. Comparing learning across these rounds allows us to test for frictions in communication and information-processing which may interfere with social learning.

Seems straightforward so far. The experiment involves several rounds, which proceed somewhat differently from each other:

Individual round. The Individual round proceeds as follows. First, the participant draws a set of balls from the urn, followed by a guess of how many red balls are in the urn. Then, they draw a second set of balls from the urn and make a second (and final) guess. All drawing and guessing is done privately, without any opportunity to share information. This round serves as a control condition - a benchmark against which we compare the other conditions.

Discussion round. The Discussion round differs from the Individual round in that, for each participant, their teammate’s draws - accessible through a discussion - serve as their ‘second’ set of draws. Each person first makes one set of draws followed by a private guess, exactly as in the Individual round. Next, the couple are asked to hold a face-to-face discussion and decide on a joint guess. After their discussion and joint guess, each person makes one final, private guess.

By comparing the final private guesses in the individual round and the discussion round, Conlon et al. can test whether learning your teammate’s information through a discussion is just as good as receiving the information directly yourself. There are two further rounds as well:

Draw-sharing round. This round is identical to the Discussion round except that, after participants receive their first set of draws and enter their first guess, they are told their teammate’s draws (both number and composition) directly by the experimenter, e.g. “Your spouse had five draws, of which three were red and two were white.” They then make an additional private guess which can incorporate both sets of draws before moving on to the discussion, joint guess and final private guess...

Guess-sharing round. The Guess-sharing round is the same as the Draw-sharing round except that the experimenter informs each person of their spouse’s private guess (made based on their own draws only), rather than their spouse’s draws. The experimenter also shares the number of draws this guess was based on, e.g. “Your spouse had 5 draws and, after seeing these draws, they guessed that the urn contains 12 red balls.”

By comparing the final private guesses in the individual round and the draw-sharing round, Conlon et al. can test whether the identity of who learns the information matters (aside from who shares that information). By comparing the final private guesses in the discussion round and the draw-sharing round, Conlon et al. can test the extent to which communication frictions affect decision-making (since there are no frictions in the draw-sharing round, because the information is shared by the experimenter, and the teammates cannot discuss at all). Comparing the draw-sharing round and the guess-sharing round allows Conlon et al. to test whether beliefs about the competence of the teammate matters. Finally, having teams made up of spouses or made up of mixed-gender strangers or same-gender strangers allows Conlon et al. to see if spouses share information (or act on information) differently than strangers do.

Having run these experiments, Conlon et al.:

...first compare guesses in the Individual and Discussion rounds, played in randomized order. Husbands put 58 percent less weight (p<0.01) on information their wives gathered - available to them via discussion - than on information they gathered themselves. In contrast, wives barely discount their husband’s information (by 7 percent), and we cannot reject that wives treat their husband’s information like their own (p=0.61). The difference in husbands’ and wives’ discounting of each other’s information is statistically significant (p=0.02).

The lower weight husbands place on their wives’ information is not due of a lack of communication from wives to husbands. In another experimental treatment - the Draw-sharing round - husbands put less weight on their wife’s information even when it is directly conveyed to them by the experimenter (absent any discussion). In this case, husbands discount information collected by their wives by a striking 98 percent compared to information collected by themselves (p<0.01), while wives again treat their spouses’ information nearly identically to their own. Lack of communication between spouses or husbands’ mistrust of (say) wives’ memory or ability thus cannot explain husbands’ behavior. Rather, husbands treat information their wives gathered as innately less informative than information they gathered themselves. In contrast, wives treat their own and their husbands’ information equally.

The guess-sharing round doesn't appear to reveal too much of interest, with results that are similar to the draw-sharing round. Finally, comparing the results from teams of spouses with the results from teams of strangers, Conlon et al. find that:

In both mixed- and same-gender pairs, men and women both respond more strongly to their own information than to their teammate’s. Thus, the underweighting of others’ information appears to be a more general phenomenon. Husbands treat their wives (information) as they treat strangers; wives instead put more weight on their husband’s information than on strangers’ information.

There is a huge amount of additional information and supporting analysis in the working paper (far too much to summarise here), so I encourage you to read it if you are interested. Conlon et al. conclude that there is:

...a general tendency to underweight others’ information relative to one’s ‘own’ information, with a counteracting tendency for women to weight their husband’s information highly.

Now of course, this is just one study that begs replication in other contexts with other samples. However, I'm sure there is a large section of the population who would find that conclusion meets their expectations (and/or their lived experience).

[HT: Markus Goldstein at Development Impact]

Gender differences in fields of specialisation within economics

Following on from yesterday's post about simple information interventions to close the gender gap in economics, the same AEA Papers and Proceedings issue had a couple of articles on gender differences in the field of specialisation within economics, which are worthwhile considering. The first was this article by Nicole Fortin, Thomas Lemieux, and Marit Rehavi (all University of British Columbia), and looks at:

...the placement outcomes of nearly 5,000 economics PhD graduates from 82 US and Canadian institutions ranked among the top 200 in the RePEc rankings who have sought employment through Econ Job Market... between 2010 and 2017...

Their decomposition analysis first shows differences in the types of jobs that PhD graduates of different genders choose, specifically:

...that the largest and most significant gender differences are for research positions.. the combination of assistant professors and CB/MDB positions, where we find 61.4 percent of men versus 55.3 percent of women. Conversely, 11.8 percent of women versus 9.7 percent of men accept postdoctoral positions, and 24.2 percent of women versus 20.5 percent of men are in other nonacademic positions.

There are also substantial differences in the field of specialisation, where Fortin et al. find that:

Women appear to be crowding into a few fields where there are limited positions available. Women are concentrating in the traditionally female fields of health, education, and welfare (I) and labor and demographic economics (J): 28.9 percent of women versus 16.7 percent of men are in these fields...

Women are underrepresented in fields such as macroeconomics, monetary economics, and finance, which have better employment prospects because of more positions outside of academia.  

That isn't too surprising, and accords with earlier research, and also with the second article, by Eva Sierminska and Ronald L. Oaxaca (both University of Arizona), which used data on the fields of nearly 7500 graduate students and nearly 4500 faculty from across 385 universities in the US and overseas. First, looking at students who defended their PhD between 2009 and 2018, they find:

...gender differences in specializations are statistically significant in four fields. Women are more likely to be present in labor/health by 13 percentage points and less likely to be in econometrics, micro, and macro/finance by 2, 3, and 6 percentage points, respectively.

Then, looking at faculty, and separating their results by the year that the academics' PhDs were granted, they find that:

Among those who have completed their PhD prior to 1989, gender differences in specializations are statistically significant in five broadly defined primary fields: econometrics, labor/health, macro/finance, development/growth/international, and other. Women are underrepresented in econometrics and macro/finance (by 2 and 9 percentage points, respectively), and they are overrepresented in labor/health (by 14 percentage points) and other fields (by 3 percentage points). In this earlier period, women were also underrepresented in development/growth/international by 2 percentage points (in later cohorts, they were over represented).

Gender differences in these fields continue to be significant for more recent PhD graduates (except in other). In the group of academic economists who completed their degrees during 1990-2003, an underrepresentation for women occurs in micro by 4 percentage points. The gaps in labor/health and macro/finance slightly diminish (by 1 percentage point).

In the most recent sample of academics, those who completed their degrees between 2004 and 2016, the gaps continue to exist in the same fields and in addition occur in industrial organization (2 percentage points).

So, there are clear gender differences in field of specialisation within economics, and those differences have been persistent over time. What isn't clear from these papers is what contributes to these persistent differences. To what extent is this simply a difference in research preferences (and research topic preferences) between men and women? To what extent is it driven by the fields of specialisation in PhD advisors, and gender matching in advising (more on a related topic in a future post)? Are there differences in the culture of the different sub-fields that make a difference (e.g. see here)? Answering these questions would help us to understand these differences in fields of specialisation better.

Women are more competitive when they can be more pro-social

There is an array of research that demonstrates that men are more competitive than women (and boys are more competitive than girls; e.g. see here). This effect has been amply demonstrated in laboratory experiments for example (e.g. see here). In a recent article published in the journal PLoS ONE (open access, with a non-technical summary on The Conversation), Alessandra Cassar (University of San Francisco) and Mary Rigdon (University of Arizona) investigated whether the gender gap in competitiveness in laboratory contexts arises from the payoff structure of those experiments. Cassar and Rigdon argue that:

...women may be just as competitive as men, if the incentives involved reflect the social environment.

In their experiment, which was undertaken with 238 undergraduate students at Chapman University; University of California, Santa Cruz; and Simon Fraser University, Cassar and Rigdon:

...focus on one such scenario in which the prize for winning a tournament includes a social dimension: Winners have the option to share some of the prize with one of the losers. This prosocial option, known to the participants ahead of the competition, may appeal to women who are motivated to gain control of the distribution of resources or to repair social connections post-competition.

Research participants were:

...randomly assigned to one of two treatments: Baseline or Dictator. Each treatment consists of three rounds of a real effort task, the matrix search, under varying payment schemes: a piece rate per correct answer (round 1), a mandatory tournament where all subjects experience the competitive environment (round 2), and a choice between being paid according the piece-rate scheme or the tournament scheme (round 3).

Cassar and Rigdon then look at differences in the choices made in Round 3 of the experiment, between those who won were able to share some of the proceeds of the task with others (the 'Dictator' treatment, named after the 'Dictator Game'), and those who didn't get the option to share (the 'Baseline' treatment). The outcome is well-summarised by Figure 1 in the paper (which shows the proportion of male and female participants who chose the tournament rather than the piece rate in Round 3 of the experiment):

Women are clearly more competitive (more willing to choose the tournament rather than the piece rate) in the Dictator treatment than in the Baseline treatment. This difference is highly statistically significant, and in further analysis:

...controlling for these differences in individual abilities (using performance in the mandatory round 2 tournament), risk preferences, and beliefs explains some of the gender gap in competitiveness, but leaves the interaction effect of gender and treatment reported in model 1 largely unchanged and equally significant...

So, changing the context of the reward scheme can change the incentives such that women are just as competitive as men. The question now is, how can this be translated to real-world contexts (such as tournaments in the labour market or in education)?

[HT: The Conversation]

Read more:

• Do single-sex schools make girls more competitive?

Gender differences in answering multiple choice in a high-stakes test

Back in October, I posted about gender differences in multiple choice answering, which was based on research that used data from the PISA survey of high school children. The results demonstrated that male students perform better than female students, which is a common feature of multiple-choice tests (see the links at the bottom of this post for more). The research also provided some suggestive evidence that confidence in answering was important, along with stereotype threat, as I noted in this 2019 post. Related to confidence, part of the difference in performance between male and female students depends on differences in the propensity to leave some questions blank. Students that are less confident (who are relatively more likely to be female students than male students) are less likely to leave an answer blank.

Now, PISA is a 'low-stakes' test, in the sense that the results don't matter for the students at all. So, there is no negative consequence to leaving a question unanswered. That isn't the case in a high stakes examination, where guessing may come with a positive net payoff (if there is no penalty for a wrong answer), or may offer no net advantage (if there is a penalty). In that case, students may differ in whether they leave questions blank, depending on their confidence and their degree of risk aversion.

The propensity of male and female students to leave questions blank is investigated in this recent article by Perihan Saygin and Ann Atwater (both University of Florida), published in the journal Economics of Education Review (sorry, I don't see an ungated version online). They use data from the Turkish OSS, which is the main college admissions examination. This examination has an interesting feature that it has several sections, and those sections have different weights for different students. As Saygin and Atwater explain:

The Turkish high school curriculum features high school students being split into tracks of their choosing in their second year. The tracks are Science-Mathematics (Quantitative), Literature-Mathematics (Equally Weighted), Social Science-Literature (Qualitative), Foreign Languages, and the Arts. These fields line up with the topics covered on the ¨OSS. It has a total of four core sections: social science (which covers history, geography, and philosophy), science (biology, chemistry, and physics), mathematics, and literature... For each subject, the exam has a lesser difficulty section and a higher difficulty one. Thus, there are 8 sections in total on the ¨OSS...

The section weights then depend on which track the students were in during high school. So, a student in the quantitative track will have a higher weighting on the maths sections, but a lower weight on the social science section, whereas a student in the social science-literature track has the opposite. This allows Saygin and Atwater to investigate how differences in the stakes associated with particular sections affects the difference in propensity to leave questions blank between male and female students. Based on a sample of 1792 randomly selected OSS participants taking the OSS examination for the first time, they find that:

...not only is the gender difference in tendency to leave questions blank largest on sections that cover mathematics, but that this difference is only significant for the test takers on the track that places the most emphasis on these sections... We also provide evidence that this gap is larger on more difficult sections of a given subject despite these sections being weighted equally to the lower difficulty sections in score calculations.

Saygin and Atwater argue in a number of places in their paper that their results demonstrate that risk aversion isn't playing a role. However, when you see the results summarised as they are above, it is hard to draw any other conclusion. If a student is worried about the risk associated with guessing, then that risk is highest on the sections of the examination where the stakes are highest. Unfortunately, Saygin and Atwater don't have any measure of risk aversion. They do have measures of confidence, and looking at those they find that:

...a positive self-assessment on a given subject is related to skipping behavior on that subject and explains part of the gender differences in tendency to leave questions blank. In addition to this, we provide evidence for gender differences in self-assessment in a given subject conditional on the performance on the corresponding test section. Male test-takers are more likely to report a positive self-assessment than their test performance would suggest in math, science, and social science while this gender difference is inverted in literature. This variation in reported self-assessment across subjects matches the pattern of the observed gender differences in tendency to leave questions blank across subjects.

So, overconfidence explains at least some of the difference in leaving questions blank. The inability to convincingly eliminate risk aversion, or the interaction between risk aversion and confidence, leaves the question of the mechanisms underlying this behaviour still open.

Also, the sample that Saygin and Atwater use is somewhat idiosyncratic. The 1792 final sample is drawn from a larger random sample of nearly 10,000 OSS students. As far as I can tell, the smaller sample arises when they eliminate students that are attempting the OSS examination for the second or subsequent time. That suggests that around 80 percent of students in the OSS make multiple attempts before they get an examination ranking that they are satisfied will gain them entry into a good Turkish university. Clearly, there is a risk of selection bias in the sample in this research that is not accounted for.

Anyway, this paper modestly contributes to the idea that confidence contributes to the gender difference in multiple choice examination performance. However, we still need more research to better understand this topic.

Read more:

• On multiple choice vs. constructed response in tests and exams
• Gender differences in multiple choice answering
• Gender differences in answering in the PISA mathematics test

Is this increasing gender equity, or gender inequity?

A post at the Dangerous Economist pointed me to this 2020 Medium article by Koen Smets (which is worth reading in its entirety):

Motor insurance in Europe forms a very interesting case study. Traditionally, insurers charged women less, because they tend to be safer drivers, and hence make fewer and smaller claims. Unlike life expectancy, the factors determining the risk here are much more linked to individual choice and behaviour. Since 2012, an EU directive forbids insurers to use gender as an element in the calculation of the premium. So, in Q4 2011, men paid on average 17% more than the overall average premium, while women paid 20% less. In 2018, that difference with the overall average premium had shrunk to a 5% uplift for male drivers, and a 6% discount for female drivers. (The residual difference stems from the fact that men tend to drive more miles per year, in more powerful cars.) So, relatively speaking, the ‘gender equity’ intervention has increased the average premium for the lower-risk women by more than 17%, while it has cut it by just under 10% for the higher-risk men. Is this an improvement? It’s not so sure. [sic]

Insurers charge higher motor vehicle insurance premiums to male drivers, because male drivers cost the insurers more. Male drivers tend to drive more kilometres, and have more severe accidents (see also here). It is reasonable for insurers to charge a higher premium to a more costly segment of the population, especially where those drivers are more costly because of their own behaviour (men could be lower cost to insurers, if they drove differently).

Insurance is subject to an asymmetric information problem that economists refer to as adverse selection. The uninformed party (the insurer) cannot easily tell drivers with 'good' attributes (low-risk drivers) apart from drivers with 'bad' attributes (high-risk drivers). To minimise the risk to themselves of engaging in an unfavourable market transaction, it makes sense for the insurer to assume that every driver is high risk. This leads to a pooling equilibrium - low-risk drivers are grouped together with the high-risk drivers and all drivers pay the same premium, because they can't easily differentiate themselves.

Since the premium is based on drivers of all risks on average, many low-risk drivers will find the cost of insurance to be too high. They will drop out of the market. The average risk of the remaining pool of insured drivers will increase, so the insurer will need to increase the insurance premium. Medium-risk drivers may then find the premiums too high, and drop out of the market. So, insurers raise premiums again. And so on, until the market fails because only the riskiest drivers would be left, and the insurer surely doesn't want to insure them!

One way of avoiding this adverse selection problem is for insurers to try to reveal how risky a driver each insurance applicant is. That way, we would have a separating equilibrium, where high-risk drivers pay higher premiums, and low-risk driver pay lower premiums. When the uninformed party tries to reveal private information (like how risky a driver an insurance applicant is), we refer to this as screening. In this case, the insurer uses various characteristics of the insurance applicant to estimate how risky they are likely to be. These characteristics might include their insurance history, past driving behaviour, the type of car they are insuring, and their demographic characteristics (including age and gender).

By imposing a law that equalises insurance premiums for men and women, the government is essentially telling insurers that they can no longer use gender as a screening tool for determining which drivers are higher risk. In the absence of that information, the insurer is a bit less informed than before. It moves things back towards the pooling equilibrium (but not all the way, because insurers still know the other details of the applicants). At the margin, insurers will now tend to over-estimate the risk of female drivers, and under-estimate the risk of male drivers. The consequence of this is that the premium for riskier male drivers becomes lower than it would have been without the law, and the premium for female drivers becomes higher than it would have been without the law. Essentially, relatively safe female drivers are cross-subsidising relatively riskier male drivers.

Wait! Wouldn't the intention of the EU directive have been to increase equity between female and male drivers? If you have a policy that equalises insurance premiums for male and female drivers, but in so doing makes male drivers better off and female drivers worse off, is that actually increasing gender equity, or decreasing gender equity? It would be interesting to know whether the policy makers had thought about this at all.

Read more:

• Car insurance, adverse selection and gender

Gender differences in answering in the PISA mathematics test

I've previously written a couple of posts on gender differences in multiple choice answering (see here and here). The key points of the research I highlighted there is that male students perform relatively better in multiple choice questions than female students. However, a good answer as to why male students perform better still eludes us. Here's what I said in my 2019 post:

Maybe it is that female students don't respond well to high pressure or competitive situations (and multiple choice questions are reasonably high pressure). Women are more risk averse than men, so maybe it is related to that? Female students might be more likely to skip questions in order to avoid the risk of losing marks... Also, men are more overconfident than women, so maybe it is related to that? Again, male students might be less likely to skip questions, because they are less likely to be unsure they have the right answer.

So, with that question still open, I was interested to read this job market paper by Silvia Griselda (Bocconi University) on the gender gap in mathematics. Griselda used the PISA tests from 2012 and 2015, which included around 500,000 students in total, from over 60 countries. Interestingly, I hadn't realised that the PISA test formats varied, with different students facing different numbers of multiple choice, closed response questions (which require a very simple, short answer, e.g. a single number), and open response questions (which require a more detailed answer, e.g. an explanation of how the answer was derived). So, Griselda compares the performance on the test between male and female students, based on the proportion of mathematics questions that were multiple choice. She first notes that:

Both in the PISA 2012 and 2015 tests, boys perform better than girls in all formats of mathematics questions. Yet, the gender difference in performance is significantly bigger in multiple-choice questions.

Then in her main analysis, Griselda finds that:

...an increase in the proportion of multiple-choice questions by 10 percentage points differentially reduces girls' scores by 0.031 standard deviations compared to boys in 2012, and by 0.021 in 2015. The effect of multiple-choice questions on the gender gap in performance is not small... This effect is comparable to a decrease in teacher quality of one-quarter of a standard deviation... or an increase in a class size of one student...

There also appear to be spillovers, whereby female students also perform worse on both closed response and open response questions, when more of the mathematics questions are multiple choice. She then further exploits the data from the 2015 PISA, which was computerised and recorded the time that each student spent on each question. Categorising students as 'inattentive' if they spent too little time on three or more questions (defined as being shorter than 10 percent of the time taken on average for that question in the student's country), and/or if they skipped three or more questions (despite having five or more minutes left when they finished the test). She finds that:

...boys are significantly more likely than girls to be identified as inattentive students (the proportion of inattentive boys is 9.38%, while the proportion of inattentive girls is 8.51%, and the difference is statistically different from zero)...

...when the proportion of multiple-choice question features [sic] in the exam increases, girls become differentially more disengaged than boys... This means that a 10 percentage point increase in the proportion of multiple-choice questions can reverse the gender gap in student engagement level.

So, when there are more multiple choice questions, female students are more disengaged (inattentive) and that explains their worse performance in multiple choice. Griselda takes things further though, showing that:

The proportion of multiple-choice questions received has a negative and significant effect on the gender difference in performance among students with a low level of confidence and self-efficacy. On the contrary, there is not a significantly different effect of the proportion of multiple-choice questions receive among high confident [sic] and high self-efficacy students...

The proportion of multiple-choice questions has a negative and significant effect on female performance only among students whose mother is not employed in STEM-related occupations. The marginal effect of the proportion of multiple-choice is not statistically significant among students whose mothers work in STEM-related occupations.

That would seem to support the idea of stereotype threat, as noted in my 2019 post. However, there is one problem that I see with this paper, and that is that the results only seem to hold for mathematics, and not for science (or for reading). If multiple choice is a problem for female students more than male students, then that difference should be apparent for all domains, and yet Griselda finds:

The proportion of multiple-choice questions in reading does not affect males and females performance, while the proportion of science multiple-choice question has an unclear effect on students' performance in science.

I would have been interested to see the same sort of analysis based on mothers in STEM-related occupations, for the science results (comparable results for confidence wouldn't be able to be done, as only confidence with mathematics was asked in 2015).

So, these results again demonstrate that male students perform better than female students in multiple choice questions. They provide further suggestive evidence that this might relate to confidence and stereotype threat. However, given that they appear to hold for mathematics and not for science, they are not conclusive to me. We need more research like this, especially trying to identify the mechanisms that are driving the headline differences. Without understanding the mechanisms, we won't be able to adequately address the problem.

[HT: This article by Griselda in The Conversation, back in May]

Read more:

• On multiple choice vs. constructed response in tests and exams
• Gender differences in multiple choice answering

More on comparative advantage and the gender gap in STEM

Back in 2018, I wrote a post on comparative advantage and the gender gap in STEM, based on two research papers, where I noted:

So, even though female students may be better than male students in STEM subjects at school, we may see fewer of them studying those subjects at university (let alone taking them as a career), because female students are also better in non-STEM subjects at school, and they are better by more in non-STEM than in STEM, compared with male students. Economists refer to this as the students following their comparative advantage. Female students have a comparative advantage in non-STEM, and male students have a comparative advantage in STEM subjects.

In this post, I want to build on that by summarising two other research papers. The first is this article by Thomas Breda (Paris School of Economics) and Clotilde Napp (Paris-Jourdan Sciences-Economiques), published in the journal Proceedings of the National Academy of Sciences in 2018 (open access). Breda and Napp used data from the 2012 wave of PISA, covering some 300,000 15-year-old students across 64 countries. They showed that, in the PISA data:

...boys outperform girls in math by about 10% of a SD... In contrast, girls outperform boys by about a third of a SD in reading. Together, these observations suggest that girls have a comparative advantage in reading, something that appears more strikingly when we look at the gender gap in the difference between math and reading (MR) ability....

Breda and Napp then construct a measure of students' intentions to pursue maths-intensive studies and careers. They found that:

The gender gap in intentions cannot be explained by differences in math ability across genders...

That makes a lot of sense, because simply being good at maths isn't enough to encourage students to follow through on math. That depends on their comparative advantage - that is, is the student good at maths but better at other disciplines? When looking at the relationship between intensions and the difference between maths and reading (MR), Breda and Napp found that:

...the gender gap in intentions to pursue math-intensive studies and careers disappears almost entirely when one controls for individual-level differences in ability between math and reading.

In other words, the intention to study maths is more associated with the difference between maths and reading ability than it is by maths ability alone. On top of that, the difference between maths and reading ability does a better job of explain intentions than self-perceived maths ability.

The second paper (still a working paper), by Sofoklis Goulas (Stanford University), Silvia Griselda (University of Melbourne), and Rigissa Megalokonomou (University of Queensland), takes the concept of comparative advantage one step further. Their concept of comparative advantage is not just the difference between a high school student's average performance in STEM and non-STEM subjects, compared with the differences for other students in their class. What Breda and Napp refer to as comparative advantage, Goulas et al. refer to as absolute advantage. I think I prefer the Goulas et al. conception, because it more clearly conforms to what we think of as comparative advantage in a trade context - comparing opportunity costs of production between countries is analogous to comparing relative performance in STEM/non-STEM between students. A within-student comparison (like Breda and Napp) is more like a within-country comparison of production costs, i.e. absolute advantage.

Anyway, Goulas et al. have data from over 70,000 Grade 10 Greek students from 123 high schools over the period from 2001 to 2009. One of the interesting aspects of their data is that these students are assigned to classes automatically based on their surname (alphabetically). This means that they are essentially randomly allocated to classroom peers, which is important in overcoming selection bias (as I noted in Sunday's post on peer effects). Their measure of comparative advantage was the in-class ranking for each student, in terms of the difference in their average grades between STEM (algebra, physics, and chemistry) and non-STEM (modern Greek, Greek literature, and ancient Greek). Class ranking is a measure of relative performance in the class, for a group of students that it would be natural for students to compare themselves to (and for whom they probably have good information about).

Using this measure, Goulas et al. found that:

Females perform, on average, significantly higher than males in almost every subject... females' over-performance are even higher in non-STEM (=1.594) compared to STEM (=0.349)... Combining these, females have a lower comparative advantage in STEM subjects compared to males (0.409 for females and 0.487 for males).

Making use of their measures of absolute advantage (difference in average grades) and comparative advantage (within-class rank), they then look at the effects on future applications to STEM programmes in Grade 11. Focusing on the comparative advantage results, they found that:

The estimated coefficient of comparative STEM advantage is not significant for males but it is significant and equal to 0.19 for females (=0.030+0.161). This means that females who are ranked at the top of their classroom distribution in grade 10, are roughly 19% more likely to enroll in a STEM track in grade 11 than females who are ranked at the bottom of their classroom distribution, ceteris paribus...

Our findings suggest that between 4 and 6 percentage points of the 34-percentage-point gender gap (or 12-18%) in initial STEM specialization in high school are attributable to the influence of the comparative STEM advantage.

Looking at longer-term outcomes, Goulas et al. also find that comparative advantage in STEM in Grade 10 leads to a higher probability of applying to a degree-level STEM programme in university. The difference between being at the top and being at the bottom of the classroom distribution leads to a 10 percent higher likelihood of applying to a STEM degree programme. These results all appear to hold when comparing only with classmates of the same gender, when comparing at the school level rather than the class level, when changing the definition of what counts as STEM or non-STEM, and in a number of other robustness checks.

So, why does comparative advantage have such a large effect for female students, but not male students? Goulas et al. pose two mechanisms. First, they suggest lower monetary returns for women in STEM-related fields, which reduces the returns to STEM-related study. However, this is hard to reconcile with the Breda and Napp paper, which notes that the gender wage gap in STEM-related occupations is lower than for non-STEM-related occupations. The second mechanism is different preferences for STEM occupations. STEM occupations tend to be more competitive, and there is a gender gap in competitiveness (see this 2014 post, for example). Societal and environmental influences (including parents), and a lack of role models (which has been suggested as an important factor in female students not studying economics) could also contribute to this.

Coming back to the Breda and Napp article, they have an interesting suggestion on how to close the gender gap in enrolments, given the high contribution of comparative advantage:

As the gender gap in reading performance is much larger than that in math performance, policymakers may want to focus primarily on the reduction of the former. Systematic tutoring for low reading achievers, who are predominantly males, would be a way, for example, to improve boys’ performance in reading.

Redirecting education resources towards boys in order to reduce the gender gap in STEM would no doubt strike many people as counter-intuitive. It also comes with ethical issues. If STEM-related occupations are higher paying, then redirecting (male) students so that they instead study non-STEM-related subjects doesn't necessarily strike me as morally unambiguous solution. Breda and Napp make some noises in that direction but avoid being explicit about the ethical problems, while Goulas et al. more-or-less ignore the policy prescription and associated ethical issues. However, sooner or later, if we are serious about addressing the gender gap, we will have to engage with the ethical implications.

[HT: Marginal Revolution for the Breda and Napp paper; The Conversation for the Goulas et al. paper]

Read more:

• STEM-readiness and the gender gap in STEM at university
• Comparative advantage and the gender gap in STEM

What Jeopardy and Junior Jeopardy can tell us about gender differences in risk taking

Game shows are fun, and funny. As a bonus, they can provide a window into the contestants' decision making in a setting where the rules are known (and if you haven't already seen it, you should check out the show Golden Balls that I blogged about here). And they can provide data that economists can exploit to understand that decision-making.

That is exactly what this 2017 article (ungated) by Jenny Säve-Söderbergh and Gabriella Sjögren Lindquist (both of Stockholm University), published in The Economic Journal, does. Säve-Söderbergh and Sjögren Lindquist (hereafter SSSL) use data from the Swedish edition of the game show Jeopardy and Junior Jeopardy to investigate gender differences in risk taking, and the influence of the gender composition of the other contestants. Specifically, they are looking at whether women (and girls) make different decisions when competing against men (and boys) than when competing against other women (and girls). They also look at whether the differences are the same for adults (in Jeopardy), as for 10-11 year old children (in Junior Jeopardy).

Specifically, they look at what happens when the contestants receive a Daily Double, where they have the option to wager some of their current score on getting the answer (or, since this is Jeopardy, the question) right. Using data from 2000 and 2001 (206 shows of Jeopardy, with 449 contestants) and from 1993-2003 (85 shows of Junior Jeopardy, with 222 contestants), they find that:

...there is no gender gap in wagering among children, in contrast to the results for adults. This result is robust to controls for absolute performance, the difficulty level of the questions, experience, relative performance and performance feedback, in addition to whether children shared the game earnings with their classes. Our second finding is that male and female risk taking differ with age in different ways: whereas girls wager more than women, boys wager less than men.

That in itself is interesting. Girls aged 10-11 years are more risk-takers than boys of the same age, but this reverses among adults. It has been established in many studies that men are less risk averse than women (although those findings are contested), but girls being less risk averse than boys is a surprise.

SSSL then go on to find that:

...female behaviour is sensitive to social context. In particular, despite the high-stakes setting and the lack of strategic advantage created by providing incorrect answers, girls perform worse (answering the Daily Double incorrectly more often and winning less often) and employ less gainful wagering strategies when they are randomly assigned a group of boy opponents compared with when they are randomly assigned a same-gender group of opponents or a mixed-gender group of opponents... Conversely, women wager less if they are randomly assigned a group of male opponents... The performances of boys and men do not change with the social context...

So, essentially boys and men don't seem to care who they are playing against, but girls and women do. And it seems that the social context affects girls more than adult women, since it impacted girls' success in getting the Daily Double question right. SSSL interpret this as potentially showing stereotype threat (where girls perform worse than boys when there is a belief that on average, girls will perform worse than boys). To support this, the authors note that their results may:

...reflect feelings of intimidation in the presence of boys that therefore causes girls to be prone to making mistakes.

It is definitely concerning if girls as young as 10 are being affected by stereotype threat. You could put this down to being just one study in one particular (and fairly unique) context. However, it seems that there is a long history of studies that have identified stereotype threat among children (see here for an overview). If we're concerned about gender gaps among adults, and among university students, then it appears that solutions need to start from a much younger age.

Does studying economics make you sexist?

Economics has a gender problem. If you don't believe it, just watch this webcast of the session "How can economics solve its gender problem?", from last week's AEA meeting in Atlanta. Or the several posts I have written on the gender gap (see the list at the end of this post).

The question of whether economics is sexist is pretty much settled (the answer is, yes it is). The relevant question now, is why? Is it because studying economics somehow makes students more gender-biased (maybe because they encounter fewer female peers, or fewer female faculty members)? Or is it because students who are already more gender-biased (perhaps they have more socially conservative views) are more likely to study economics?

Another paper presented at the AEA meetings (UPDATE: final published paper here), by Valentina Paredes (Universidad de Chile), Daniele Paserman (Boston University), and Francisco Pino (Universidad de Chile) provides an initial answer to the question of whether studying economics makes students more sexist. They used survey data from 3228 Chilean students, of which about 26% were enrolled in 'Business and Economics'. Their sample includes both first-year students, and senior undergraduate students (Years 2 to 6), so they are able to see how gender attitudes differ between economics and other students, and how this difference changes between those just starting out as students, and those who are late in the university education (this is what economists refer to as a 'difference-in-differences' research design). They use several measures of gender-biasedness, including a composite score of all of their various measures. They report that:

For eight of the nine individual measures, we find that B&E students are more gender biased than non-B&E students, and five of the differences are statistically significant at the 5% level (in the specification with controls). The bias of B&E students appears to be particularly pronounced in the IAT-career score, hostile and benevolent sexism, and the gender roles-normative measure.16 The raw gap in the aggregate gender-bias score 0.19 standard deviations, and it falls to 0.14 standard deviations after inclusion of controls. In other words, the gap between B&E and non-B&E students is about one quarter as large as the gap between men and women. This is a sizeable difference.

So, there is more gender bias among Business and Economics students than among other students. But, is this a result of studying economics, or were they more biased to begin with? Paredes et al. report that:

The difference-in-differences estimator is positive for all measures (in the version with controls), but never statistically significant. Therefore the evidence in support of the training hypothesis is rather mixed.

I wouldn't call that evidence mixed. I'd call it unconvincing. It appears that students choosing to study Business and Economics are already more biased before they start their studies. However, there are some interesting gender differences:

For women, there is some evidence of selection: economics students are more gender-biased upon entry, and the gap increases at most moderately in subsequent years. For male students, on the other hand, we observe a moderate-sized but insignificant gap at entry, a large and statistically significant gap among upperclassmen, and a large and statistically significant difference-in-differences estimate.

So, it appears that studying economics might increase gender bias, but only for male students. Paredes et al. go on to look at some possible mechanisms that might explain the difference. The proportion of female peers doesn't explain it, and neither does differences in political ideology. However:

The share of female faculty has a strong and statistically significant association with the gender bias score for both first year and upperclass male students, but is almost unrelated to female students’ score. The difference-in-differences estimate for male students drops from 0.175 to 0.105, implying that about 40% of the B&E-non-B&E gap for male students can be explained by differential exposure to female faculty.

A substantial proportion of the effect of studying economics on gender-biased attitudes among students can be explained by differences in exposure to female faculty. In other words, this suggests that having more female faculty is a good thing for reducing gender bias in economics. However, as noted in the webcast I linked above, this leads to a chicken-and-egg problem - the solution to reducing gender bias is to have more female economists, and to have more female economists, we need less gender bias. And given that the key difference is between first-year students and older students, we need more female faculty teaching introductory economics (am I talking myself out of my current teaching allocation?).

Of course, there are some caveats to the Paredes et al. paper. Chile has a huge gender gap problem, as noted in the paper:

The raw gender gap in monthly wages is 31.7%. Even among full-time employees, the gender gap in hourly wages is 19.4%. The United Nations Gender Inequality Index ranks Chile 65th out of 188 countries (the U.S. is ranked 43), while the World Economics Forum Gender Gap Index ranks Chile 63th of 144, and only 117th in the Economic Participation and Opportunity subcategory.

That might contribute to the results in a way that makes them less representative for other countries where gender gaps are not quite as pronounced. There is also the problem that there is a gender bias only for female first-year students (but not males), which leads to the statistically insignificant effect overall when both groups are combined. There doesn't seem to be a good explanation for why female students would exhibit gender bias on entry to university, but not male students. So, there is definitely scope for more of this type of research.

[HT: Marginal Revolution]

Read more:

• Why we should care about the gender gap in economics
• Justin Wolfers on why we should care about the gender gap in economics
• Female representation in economics
• CORE on 'missing women in economics'
• More on the gender gap in economics
• The gender gap in economics

Gender differences in multiple choice answering

In a post in 2017, I discussed multiple choice and constructed response questions in exams. One of the key points of that post was to highlight the gender difference in performance in multiple choice questions (female students do worse, but they do better on constructed response questions). It turns out that result is fairly common in the academic literature, but the reason why female students don't perform as well as male students on multiple choice questions is less clear. Maybe it is that female students don't respond well to high pressure or competitive situations (and multiple choice questions are reasonably high pressure). Women are more risk averse than men, so maybe it is related to that? Female students might be more likely to skip questions in order to avoid the risk of losing marks [*]. Also, men are more overconfident than women, so maybe it is related to that? Again, male students might be less likely to skip questions, because they are less likely to be unsure they have the right answer.

So, I was quite interested to read this 2017 article by Gerhard Riener and Valentin Wagner (both Düsseldorf Institute for Competition Economics), published in the journal Economics of Education Review (sorry, I don't see an ungated version, but it looks like it might be open access anyway). Riener and Wagner conduct an experiment with 2060 German secondary school students across 89 classes in 25 schools, who each sat a maths test (based on the Math Kangaroo questions).

There were three levels of questions (easy, worth 3 marks; medium, worth 4 marks; and difficult, worth 5 marks). Students lost one mark for every question they got wrong. So, there is an incentive not to guess the easy questions if you don't know (because there is a 1/5 chance of getting three marks, and a 4/5 chance of losing a mark, so the expected value of guessing is -0.2 [0.2 * 3 + 0.8 * -1]). There is no incentive either way for the medium questions (the expected value is 0 [0.2 * 4 + 0.8 * -1]), and a positive incentive to guess on the difficult questions (the expected value is 0.2 [0.2 * 5 + 0.8 * -1]). So, Riener and Wagner expect students to skip more of the easy questions, and fewer of the difficult questions. It turns out that wasn't the case, since they:

...find that the number of skipped questions is increasing in difficulty.

I'm not surprised by this at all. Students don't understand expected value, so it wouldn't surprise me that they didn't realise that there was a positive expected value for guessing, for the difficult questions. And the positive expected value is a relevant consideration for a risk neutral decision-maker. Since there were only 14 questions in the test (and only 4 difficult questions), then it is relatively risky to guess on one of them, in terms of the impact on overall score.

That wasn't the main results from the paper though, which concerned the gender differences, and the experiment. The experiment was that students in some classes were rewarded, if their test score was better than their earlier mid-term result (they didn't know about the experiment until after the mid-term, so there is no risk of the students engaging in strategic behaviour). The reward (according to Riener and Wagner, a source of extrinsic incentive to do well in the test) was one of: (1) a medal, awarded in front of the rest of the class; (2) a letter to their parents, praising their good performance; (3) a "no homework" voucher, which entitled them to take a day off homework; or (4) a "surprise gift" (which was actually a combination of (1) and (2)).

They find that:

Females always tend to skip more questions than males regardless of whether they are incentivized or not. However, incentivized pupils tend to skip fewer questions than non-incentivized pupils...

...girls in our low stakes baseline treatment skip significantly more questions than boys... However, the gender gap depends on item difficulty. While girls skip as many questions as boys when items are easy... they skip significantly more questions for medium... and difficult questions... Interestingly, providing extrinsic incentives for performance, and hence increasing the stakes, closes the gender gap in skipping test items. 

So, providing an incentive closed the gender gap. They also find that the gender gap is only present in academic high schools (Gymnasium) and not in vocational high schools (Gesamtschule, Realschule, and Hauptschule). However, I'm not convinced by those results, as the vocational students sat an easier test, with fewer questions used in the analysis, which renders the results not comparable with the academic high school students.

Riener and Wagner argue that their results are:

...suggestive evidence that the gender gap could be explained by a stereotype threat. Girls in high school only skip significantly more questions if the questions are difficult, although the attractiveness of answering is higher for difficult questions than for easy questions. Further support for a stereotype threat explanation is the fact that the gender gap vanishes if the difficulty of the task is made less salient (shifting the focus of pupils to winning an extrinsic reward).

It's possible that this is stereotype threat. However, it is also possible that the types of rewards that they were offering were the types of rewards that girls value more than boys (especially given that the students got to choose their preferred reward). Perhaps the reward increased the stakes of the test for girls, but not for boys? In any case, it's hard for me to see how the offering of the reward reduces stereotype threat. So, although they were able to eliminate the gender difference in performance, I don't think this paper really helps us get to the bottom of why female students usually perform worse than male students in multiple choice questions.

*****

[*] This applies if there are negative marks for a wrong answer. It is a bit harder to argue this when there is no penalty for a wrong answer (or at least, no difference in the penalty between skipping and getting the answer wrong).

Read more:

• Female student performance in high-stakes biology exams
• On multiple choice vs. constructed response in tests and exams

The gender gap and question time in academic seminars

The sources of the gender gap in STEM (and economics) are probably many and varied. However, one potential source that has been highlighted is the lack of role models (see my post on the topic here). If female senior students don't have female academic role models, then they be less likely to pursue an academic career, and if female junior faculty don't have female senior faculty role models, they may be less likely to succeed.

A recent paper by Alecia Carter (University of Cambridge), Alyssa Croft (University of Arizona), Dieter Lukas (University of Cambridge) and Gillian Sandstrom (University of Essex) looks at one example of role modelling - the asking of questions in academic seminars (with a nice non-technical summary on the LSE Impact Blog). Specifically, they collected survey data from 509 academics (mostly in biology or psychology), as well as observational data from "247 seminars, from 42 departments of 35 institutions in 10 countries".

They found that:

...a given question after a departmental seminar was more than 2.5 times more likely to be asked by a male than a female audience member, significantly misrepresenting the gender-ratio of the audience which was, on average, equal.

So, even after controlling for the gender make-up of the audience, and the hosting institution, and the gender of the speaker, men were more than twice as likely to ask questions as women. They conclude that:

[i]n the case of academic seminars, then, the fact that our data show women asking disproportionately fewer questions than men necessarily means that junior scholars are encountering fewer visible female role models in the field.

An interesting aspect of the paper is that they tried an intervention to increase the number of questions from female audience members. They first noted that there were more questions (and more questions from women) when the time allowed for questions was longer. They then manipulated some seminar lengths to allow for more time for questions. However, the intervention had no effect on the number of questions asked by female audience members.

Having read the paper though, there was an alternative intervention they could have tried. The number of questions from female audience members also appears to be higher when a woman asks the first question (even if you exclude the first question from the analysis). So, a simpler intervention would have been to ask the seminar moderator or chair to ensure that a female audience member asks the first question (if possible).

I've just gotten back from a conference in the U.S. I can't say I noticed a gender disparity in question asking there, but then again I wasn't looking. I'll definitely be paying more attention during departmental seminars and conference sessions in future.

[HT: eSocSci]

Intergenerational transmission of gender bias and the gender gap in STEM

More male students than female students choose to study STEM subjects (and economics) at university. In a recent post, I discussed two papers that could be interpreted as suggesting comparative advantage as an explanation. That is:

...even though female students may be better than male students in STEM subjects at school, we may see fewer of them studying those subjects at university (let alone taking them as a career), because female students are also better in non-STEM subjects at school, and they are better by more in non-STEM than in STEM, compared with male students. Economists refer to this as the students following their comparative advantage. Female students have a comparative advantage in non-STEM, and male students have a comparative advantage in STEM subjects.

However, there are other competing explanations. Students' choices are affected by their parents and by their peers. So, if their parents hold biased views about whether studying a STEM subject is suitable for female students (or if their peers do), then female students may be less likely to study STEM subjects. A recent working paper by Alex Eble (Columbia University) and Feng Hu (University of Science and Technology Beijing) looks specifically at the intergenerational transmission of bias (from parents to children).

Eble and Hu use data from 8,912 Chinese students in 215 seventh and ninth grade classrooms, who were routinely randomly assigned to classrooms. That means that each student's peer group (within their classroom) was randomly assigned. The key variables that Eble and Hu are interested in are the bias in the views of each student's own parents, and similar bias in the views of each student's peers' parents. They then look at how those two biases are related to students' own bias, aspirations, and academic performance. They find that:

...a one standard deviation increase in peer parents’ bias causes a 4.2 percentage point (8%) increase in the likelihood that a child will hold the bias... This transmission of bias appears to occur for both girls and boys in roughly the same manner... we calculate that going from roughly 25 percent of peers’ parents being biased to 75 percent of peers’ parents being biased generates an 18.9 percentage point (34%) change in the likelihood that a child will also hold that bias...

Children whose parents hold anti-girl bias are 29 percentage points (52%) more likely to also hold that bias, and again the transmission appears to hold equally for boys and for girls...

Ok, so there is intergenerational transmission of bias. How does that affect the gender gap? Eble and Hu find that:

...an increase in peer parent bias increases the gender gap... in girls’ perceived difficulty of math relative to boys’ by two percentage points, or 28 percent of the 7.2 percentage point gap between boys and girls in this variable. This pattern also holds for own parents’ bias, and the estimates are again more stark: our estimated coefficient of own parent’s bias on the gender gap is a 15.0 percentage point increase in the likelihood that a girl perceives math to be difficult, relative to the likelihood for boys. For boys, own parent’s bias is associated with a 6.1 percentage point decrease in the likelihood the child will perceive math to be difficult; the “total effect” for girls is an 8.9 percentage point increase in this likelihood...

As with perceived difficulty, our estimates of the effect of peer parents’ bias on boys’ and girls’ test scores, respectively, differ in sign. Boys appear to gain slightly (a statistically insignificant 0.05 SD increase) from a one standard deviation increase in the proportion of peers whose parents believe that boys are superior to girls in learning math. For girls, on the other hand, we estimate that a one SD increase in peer parent bias increases the gender gap - that is, reduces girls’ performance relative to boys’ - by a statistically significant 0.063 SD...

Here again the correlation between own parent’s bias and performance is much larger in magnitude - the scores of boys whose parents believe that boys are better than girls at learning math are 0.16 SD higher than for boys whose parents do not believe this, and for girls, having a parent who holds this bias pushes the child’s test score down, relative to boys’ scores, by 0.28 SD...

In other words, they find evidence that the gender gap in perceived difficulty of maths and in maths test scores can be partially explained by own parents' bias and peer parents' bias. Unsurprisingly, the effect of own parents' bias is the larger of the two. Interestingly, they also find that girl peers' parents have a bigger effect on girls than boy peers' parents do. However, in amongst the results they also find a potential partial solution, at least in terms of peer parents' bias:

These negative effects, however, decrease with the number of friends the child has in her class; a child with five close friends in her class appears to be entirely immune to the negative effects of peer parent bias that we have shown throughout this study...

So, it's not all bad news. If you want to give your daughter the best chance for success in STEM, start by minimising the bias in your own views. If you're worried about the biased views of other parents, to the extent you can you should make sure your daughter has many close friends in her class.

[HT: Development Impact]