Can a simple email get students to study more economics?

This year, I started sending the top students in my ECONS101 class a 'student recognition letter', congratulating them on their performance and noting their overall grade and their ranking in the class. The purpose of this letter was not to try and sell these students on studying economics, but to provide them with some well-earned recognition, as well as a valuable signal for future employers that their A+ grade actually meant something (whereas in some papers, it clearly doesn't mean much).

I could have added a sentence or two encouraging those students to study more economics. They certainly have shown an aptitude for it. However, some years ago I did try to encourage the top students to study more economics, but it seemed like those efforts didn't have much impact. I stopped doing this in about 2018. And it seems I might have made a good choice, at least according to this forthcoming article in the journal Economics of Education Review (open access) by Olivia Edwards and Jonathan Meer (both Texas A&M University).

Edwards and Meer study the effect of a simple email from the professor to the top ten percent (or so) of the class in introductory economics at Texas A&M University:

...praising their performance and encouraging them to take more economics courses and considering majoring or minoring in economics if they were not already doing so.

Their analysis is based on 10,600 students who took the course between 2017 and 2023, of which 1802 received the encouragement email. Edwards and Meer apply a regression discontinuity design (RDD) approach to the analysis, essentially comparing students just above the cutoff mark for receiving the encouragement email with those just below the cutoff mark. The outcome variables Edwards and Meer look at are whether the student went on to study intermediate microeconomics, whether they majored or minored in economics, and whether they majored in an 'economics-adjacent field' (by which they mean agricultural economics or business). They also test whether the results differ by gender, under-represented minority (URM) status, and whether the student was first-in-family to study at university.

There did seem to be some effect on students going on to study intermediate microeconomics. For students around the cutoff point for receiving the encouragement email, there was:

...a relatively large and statistically significant discontinuity of 8.8 percentage points (s.e. = 3.3 percentage points), an increase of about 40 percent over the baseline level below the cutoff.

However, there was little evidence of an effect on majoring or minoring in economics, with both effects being statistically insignificant. Turning to whether there were heterogeneous effects based on demographic characteristics, the effect of the email on taking intermediate microeconomics did not differ significantly by gender, but the overall positive effect appears to have been concentrated among URM students and students who were first-in-family to attend university. The encouragement email had no effect on going onto an economics major or minor for any of the demographic groups.

Edwards and Meer describe the effect of the encouragement email as "modest". I would say it was disappointing, but not surprising. If Texas A&M is anything like Waikato, then once a student has chosen a particular major (or minor), it is actually quite difficult to induce them to switch, even if they later find that some other major or minor would be a better fit for them. In part, this is due to institutional barriers, such as the difficulty in navigating the change-of-enrolment process, or the necessity to take pre-requisites that students might have missed, or that the student may have 'spent' papers on a particular major that would be lost if they switch. It could also be that students simply don't like changing their minds (a 'status quo bias').

So, while a simple information intervention may not be enough to change students’ choice of major, it does seem capable of nudging some students into taking more economics. And on the plus side, sending an email doesn't cost the professor much, so even if it only encourages some additional enrolments in intermediate microeconomics, on a cost-benefit basis for the Economics Department, it is probably a net positive. Given these results, perhaps I was too hasty in stopping the explicit encouragement of my top students to study more economics? I may have to re-draft my student recognition letters for this trimester.

[HT: Tim Harford, last year]

Read more:

• Could closing the gender gap in economics be as simple as providing students with information?
• Disappointing results on simple information interventions to close the gender gap in economics

This week in research #142

Here's what caught my eye in research over the past week:

• Sacerdote, Staiger, and Tine (with ungated earlier version here) find that test score–optional policies harm the likelihood of admission for high-achieving applicants from disadvantaged backgrounds, meaning that the availability of test scores on an application can promote rather than hinder social mobility
• Armona et al. (with ungated earlier version here) develop and test a model of what is newsworthy to a media outlet
• Crossin et al. (open access if you set up a free account) find using data from a nationally representative sample that an estimated 71.1 percent of the New Zealand population think politicians should do more to keep people safe from alcohol harm, with majority support across the political spectrum
• Smit (open access) outlines the drawbacks to remote working that may have prevented greater internal migration from cities to the periphery in the Netherlands

Jeanna Smialek on why we should remember Maria Edgeworth

I have read several books now on the history of economic thought and, for the most part, women are conspicuously absent from those books. One obvious exception is Edith Kuiper's A Herstory of Economics (which I reviewed here), and there are a few bits in Steven Medema's The Economics Book (which I reviewed here). The latter brought both Harriet Martineau and Jane Marcet to my attention (and my two class AI tutors, Harriet in ECONS101, and Jane in ECONS102, are named after them).

A surprising omission from both books was brought to my attention by this New York Times article (ungated version here) by Jeanna Smialik: Maria Edgeworth. [*] According to Wikipedia, Maria Edgeworth was a novelist and an aunt of the much more famous 19th Century economist, Francis Ysidro Edgeworth. However, Smialek makes a strong case for the importance of Maria Edgeworth as one of the earliest voices in the developing field of economics, with her economics expressed within her fiction. However, Smialek also spends some efforts to explain why Edgeworth has been largely forgotten as an economist:

As the field professionalized, the second- and third-generation economists distanced themselves from the women and the fiction that had once helped to popularize and explain their ideas. Serious science, after all, could not possibly be for everyone. The economist Alfred Marshall referred to the women who tried to simplify economic doctrine as “parasites” in a footnote to his hugely influential textbook, “Principles of Economics,” first published in 1890 and popular throughout the 1900s...

Her literature was lost partly because of its clunkiness — its economics lessons made it harder to digest as lighter and more naturalistic stories became the style.

Her economics was lost for a different reason. Subsequent influential academics disparaged the ways that the early women of economics had presented the field. They bristled at being simplified, and they dismissed the idea that the women could have contributed something meaningful. 

Smialek also notes that Edgeworth provided editorial comments on Jane Marcet's Conversations on Political Economy, arguably the first economics textbook:

Edgeworth was as skilled an editor as she was a writer. She told Marcet that Adam Smith ought to be abridged, because he “needs it much,” whereas the utilitarian philosopher Jeremy Bentham was “absolutely impossible” to put in fewer words, because what he needed was “to be diluted.”

“The more amusing anecdote & illustration you can mix with your solid information the better,” Edgeworth wrote to Marcet. “It should be your object rather to sow seeds than to exhibit full grown plants — Your work should excite curiosity to go further.”

Clearly, there is more for us to learn about the history of economic thought from Maria Edgeworth's life and her writing. And we are in luck! Smialek has a forthcoming book about Edgeworth due to be released next month, titled The Invisible Hand of Maria Edgeworth. I'm definitely looking forward to that one, and you can expect a review of it here in due course (although, with my backlog of reading, that might not be until sometime next year!).

*****

[*] Smialek notes that Maria Edgeworth was brought to her attention by Robert Heilbroner's The Worldly Philosophers (which I reviewed here). Heilbroner's reference to Edgeworth clearly didn't make as great an impression on me, as I don't remember it!

The impact of a subtle wording change on measurement of worries about climate change

Consider the following statement: "The effects of climate change are too far in the future to really worry me". Would you say that you: strongly agree with that statement; tend to agree; neither agree nor disagree; tend to disagree; or strongly disagree? Now consider this alternative wording of the same statement: "The effects of climate change are too far in the future to really worry about". Would your level of agreement be the same to this second statement as to the first?

There is good reason to believe that the wording of that question matters, as shown in this new article by Lieke Voorintholt (Trier University), Adriaan Soetevent, and Gerard van den Berg (both University of Groningen), published in the Journal of Economic Psychology (ungated earlier version here). They argue that the single-word difference, from "worry me" to "worry about" may shift respondents' focus from themselves individually towards broader society. Therefore, they hypothesise that more survey respondents would report being worried when asked if they worry about the effects, compared to whether the effects worry them individually. They also hypothesise that the treatment effect would be greater for older respondents, since the "about" version prompts thinking beyond the respondent's own life expectancy.

Voorintholt et al. use data from the Innovation Panel of Understanding Society, which its website describes as "a test-bed for innovative ways of collecting data and for developing new areas of research". The survey is run in Great Britain, and their analysis is based on a sample of 2791 survey respondents from Wave 16 of the survey, conducted in 2023. Half of the respondents were randomised to be asked the "me" version of the question, and the other half were asked the "about" question. Voorintholt et al. term the "me" version the control group, because that is the wording that is used in the main Understanding Society survey. So, the treatment is simply the replacement of "me" with "about" in the wording of the statement.

Their main results are summarised in Figure 1 from the article:

When asked the "about" version, more respondents say they strongly disagree with the statement, while fewer respondents say they tend to agree with the statement. These differences can be interpreted as greater worries about climate change expressed by the treatment group than the control group. This is also supported by their quantitative analysis, which shows the average response among treated respondents is 0.18 points higher (on a 1-5 scale from strongly agree to strongly disagree) than control respondents. Voorintholt et al. find some suggestive evidence in support of larger effects for older respondents. The coefficient on an interaction term between treatment and an age dummy variable for those aged 66 and over is positive and relatively large, but is statistically insignificant.

Overall though, this research is a good reminder that even a one-word change can materially affect how survey respondents answer a question, and therefore how we measure important variables. The effect of subtle changes should not be underestimated. If you want to measure the level of concern that people express about climate change, it really matters whether respondents are asked whether climate change “worries me” or whether it is something to “worry about”.