Book review: Indigenous Economics

Hot on the heels of reading Raymond Firth's Economics of the New Zealand Maori (which I reviewed here earlier this week), I read Indigenous Economics, by Ronald Trosper. This book attempts to explain economics using an Indigenous worldview. This is incredibly ambitious, given that Indigenous groups are not all alike. However, there is enough commonality that Trosper is able to weave together a coherent text using examples from North America, South America, Australia, and New Zealand.

The common themes that, according to Trosper, underlie Indigenous economics and distinguish it from 'traditional economics' are relationality and consciousness of all beings (including nonhuman beings such as animals, and more-than-human beings such as rivers or mountains). As Trosper explains in the introduction to the book:

This book explains how a focus on relationships generates a different kind of economic theory than the mainstream approach in all its variations. Persons replace the individual in standard economics; persons become active agents in the creation of the lives they live rather than passive responders to incentives given by prices. Persons combine into relational subjects of many different types that are similar in their dependence on dialogue and reflexivity among the persons who comprise them. Relational subjects can differ in scale with corresponding differences in the modes of dialogue and interaction. Relational subjects include all beings in a landscape.

The book had both strengths and weaknesses. The key strengths include the wealth of examples that Trosper uses to illustrate his ideas. However, that strength also contributed to the weakness, which is that the book is light on developing theory and principles and explaining in depth how an Indigenous economics could be constructed from those principles. I will acknowledge though, that this criticism is really based on the book that I wanted to read, which is not necessarily the book that Trosper wanted to write. If Trosper's intended audience is mainly Indigenous readers who have already in their minds rejected traditional economics and want to see their own worldview presented as economics, then he has nailed it. And indeed, the book could have been written in this way because of relationality. But for a reader like myself, who wants to understand how an Indigenous worldview enhances our understanding of economics, or even better, how a truly Indigenous economics can be constructed from principles drawn from an Indigenous worldview, the book falls short.

As you might expect, Trosper mainly does a good job of contrasting the views of 'traditional economics' with views through an Indigenous lens, and that is where the book offers the greatest value. For instance, a large section of the book essentially explains why the western conception of 'private property', especially in relation to land, is inappropriate in an Indigenous context. This is tied very closely to the two themes:

An economic system based on relationships that include nonhumans cannot use a private property system or even a property system because to do so fails to recognize the consciousness and agency of nonhumans who enter relationships. The owner of a thing as property has control over it in a way that is inconsistent with treating it in a respectful manner.

That section was particularly strong, and yet at the same time it was underdeveloped. Trosper acknowledges a counterargument based on efficiency without taking nonhumans into account, but then flippantly explains this away, with:

These criticisms ignore the ways in which Indigenous territorial systems do have incentives to use the land well; those with leadership responsibility are expected to have solid relationships with the beings living on the land. The valuation of the land includes the relational values created through relational goods.

I agree with his counterargument here, but these ideas could have been explained in more detail and the counterargument would have been much more solid. Also, linking more explicitly to the economic theory of property rights, such as from Nobel Prize winners Oliver Williamson and Oliver Hart, would have provided either a stronger foundation, or a clearer critique.

Similarly, in the section on Indigenous entrepreneurship, Trosper doesn't engage with the idea of profit maximisation (a key assumption in mainstream economics) until near the end of the section. At that point, he reframes profit as a constraint, rather than a goal, of Indigenous business:

The need to accommodate Indigenous viewpoints by business presents an obvious issue: How can one make a profit selling things or experiences without fully giving in to market culture with its individualistic focus? Part of the answer is to distinguish between maximizing profits as a goal and achieving positive profits as a constraint. This is often stated in this manner: rather than making profits the goal of the firm, the profit requirement is changed into a constraint. How can a firm pursue other goals without profit falling into negative territory?

I feel like that statement would have been a good place to start the section on Indigenous entrepreneurship, and building out the theory of profit as a constraint rather than a goal. Again, this seemed like a lost opportunity. The book did link to ideas of social enterprise, but could have looked at some of the research on social preferences beyond the work of Samuel Bowles. I also thought that there was a strong case to include some of the ideas on identity economics from Akerlof and Kranton (whose book Identity Economics I reviewed here).

Trosper also makes some simple errors in basic economics, such as claiming that "[i]n standard economics, states produce public goods and individuals produce private goods". As my ECONS102 students would attest, this is one of my pet hates. Governments can (and do) provide private goods, and many public goods are provided privately. Similarly, Trosper creates a strawman argument when he writes that:

Standard economics focuses only on private goods and public goods (Samuelson 1954).

The fact that his supporting citation is Samuelson's textbook that is over seventy years old should tell you that this statement lacks support. It is in fact quite untrue, and Elinor Ostrom's work (which Trosper cites extensively) on common resources clearly demonstrates that the statement is unsupportable. It is also quite unnecessary to create such a strawman argument. This book shouldn't be about pitting Indigenous economics against traditional economics, but instead showing how they differ in meaningful ways.

Despite my gripes, I did enjoy reading this book, and it opened my eyes to some of the possibilities for a truly Indigenous economics that is constructed from its own principles. Someday perhaps we will see that ideal fulfilled. Unfortunately, this was not the book to do it. I’d still recommend this book for readers who want an accessible introduction to Indigenous perspectives on economics, but economists looking for a fully articulated alternative theory may come away wanting more.

What Star Wars AND Star Trek can teach us about economics

This is not a “Star Wars vs Star Trek” post. I'm non-partisan. I enjoy both Star Wars and Star Trek about equally. And it turns out that I am not alone. Last December, John Hawkins (University of Canberra) wrote in The Conversation about what Star Wars can teach us about economics. This year, Hawkins (with Tesfaye Gebremedhin, also University of Canberra) wrote in The Conversation about what Star Trek can teach us about economics.

One of the best ways to teach economics is to start with stories people already know. And people know Star Wars and Star Trek. Hawkins' articles provide many interesting examples of the economic principles, concepts, and models that appear in each universe. For example:

Indeed, the star wars started from an interplanetary trade dispute. Interplanetary trade seems based on the resources each planet has in abundance, which may be some minerals or cheap labour...

There is a lesson here, both about the gains from trade and the risks of starting trade wars...

And:

A prime example is Quark, the archetypal Ferengi who owns a popular bar and restaurant. Quark exploits his monopsony power – being the sole employer in a niche market – to underpay staff and impose harsh working conditions...

But in one episode, Quark’s employees go on strike, prompting him to use underhanded tactics to suppress collective bargaining and maintain control.

This storyline mirrors real-world labour market dynamics and the tension between capital and labour.

The two articles are limited by word count, but there are many more examples that Hawkins (and Gebremedhin) could have used, including examples of the same economic principle, concept, or model appearing in both universes. For example, there are lots of examples of supply and demand dynamics, including the spice trade in Star Wars (with wartime disruption decreasing supply and increasing the price, incentivising smuggling) and Star Trek (with conflict and station politics in Deep Space Nine changing the prices of scarce commodities, which the Ferengi benefit from). Or consider the externalities from the destruction of Alderaan in Star Wars, or the Borg incursion in Star Trek. Or public goods, such as planetary defence against the Empire in Star Wars, or shared defence in the Federation in Star Trek. Or many, many examples of game theory (such as the examples in the book Hidden Games, which I reviewed here).

Science fiction may be fictional, but there is still much that we can gain from these universes in terms of illustrating economics in action. These universes are basically case studies in economics, incentives, and institutions.

Read more:

• Why the destruction of two Death Stars might not have bankrupted the galactic economy

Specialisation trends in economics, and who’s citing economics

Do research findings in one discipline affect research in other disciplines? This seems like an important question, and one to which we hope the answer is yes. And yet, for a long time disciplines seemed to be becoming more insular, only referencing research within their own narrow fields and sub-fields. More recently, it seems to me that the breadth of citation across disciplines has increased greatly. Call it the 'Google Scholar effect', if you like. Since research has become much more searchable, relevant past research has become easier to find and to cite. If there is a 'Google Scholar effect', we'd expect to see a big change in citation patterns across disciplines starting from 2004, when Google Scholar was introduced.

So, I was interested to read this 2024 article by Sebastian Galiani (University of Maryland), Ramiro Gálvez (Universidad Torcuato Di Tella) and Ian Nachman (Brown University), published in the journal Economic Inquiry (ungated earlier version here). They conduct a citation analysis to look at specialisation trends in economics. Specifically:

...we look for patterns that indicate specialization within a field of economics research, such as a narrowing of the topics it covers in a way that other fields do not, and patterns suggesting a decrease in citations from outside the field...

Their dataset includes:

...all articles published from 1970 up to and including 2016 in the so‐called economics Top 5 journals (The American Economic Review, Journal of Political Economy, The Quarterly Journal of Economics, Econometrica, and The Review of Economic Studies) and for all articles published from 1970 up to and including 2016 in a set of non‐Top 5 general research journals (The Economic Journal, International Economic Review, Economic Inquiry, and The Review of Economics and Statistics)...

Galiani et al. categorise each article into one of four 'fields' of economic research:

...(1) applied, (2) applied theory, (3) econometric methods, and (4) theory... The criteria used to assign a paper to a field were as follows: (1) Applied articles have an empirical or applied motivation. They rely on the use of econometric or statistical methods as a basis for analyzing empirical data, although they may deal with simple models that provide a theoretical framework for the analysis... (2) Applied theory articles construct a theoretical model to explain a fact, with the empirical analysis serving as a supplementary aspect rather than the primary focus of the paper. These articles typically have limited utilization of econometric or statistical analyses, though they may employ simulations (even with empirical data) or refine other techniques to test the implications of the models... (3) Econometric methods articles develop econometric or statistical methodologies... (4) Theory articles lack an empirical section; typically, they approach a topic through modeling and extensive use of formal mathematics and logic. These articles may incorporate a numerical example or a simple model calibration using theoretical data to illustrate the proposed model or to analyze its comparative statics.

The process of assigning articles to a field involves machine learning, which I'm not going to get into here. They also use natural language processing to detect 'latent topics' based on the content of each article, where each article is assigned to exactly ten topics. They use that approach, rather than using keywords or JEL codes because the topics that they identify use similar methods, making them more similar than articles that may have the same keywords but approach the research question in very different ways.

Galiani et al. then look at the long-term trends in the topics, content, and citations (both within and outside of economics) of their large sample of 24,273 articles. The results make for interesting reading, not just in terms of the trends, but in terms of the differences between the different fields of economics:

...theory and econometric methods have shown a narrowing focus on specific research topics since the 1990s, indicating a tendency toward specialization. Theory articles have experienced a significant increase in topics related to formal mathematical proofs and game theory, while econometric methods articles have shown a pronounced rise in topics related to computational statistics, estimators' asymptotic properties, and estimators' bounds. In contrast to applied papers, these fields do not exhibit rising trends in extramural citations (i.e., citations from other disciplines) and in citations from other fields within economics research (in the case of econometric methods, it shows a declining trend). These patterns also indicate a higher degree of specialization.

Applied papers have expanded their coverage to include diverse topics. Applied articles have seen a pronounced rise in topics related to impact analysis, causal analysis, and experimental economics. Over time, applied articles began to receive a higher proportion of citations from external fields, especially from disciplines such as medicine, psychology, law, and to a certain extent, education... Overall, these patterns indicate that applied papers are becoming more multidisciplinary. The case of applied theory articles is less conclusive. While they cover a broader range of topics (similar to applied papers), there has been no significant increase in extramural citations or citations from other fields of economics research (as observed with theory articles).

The observation that applied papers have received an increasing proportion of citations from outside economics over time is particularly interesting. Applied papers are more easily cited outside of economics because they often address research questions that other disciplines are also interested in. Galiani et al. identify that applied articles in economics are often cited in the education, medicine, and psychology fields, more so than articles from other fields of economics. This pattern of extramural citations does seem to become more apparent from the 1990s onwards, which would seem to suggest that Google Scholar (which was introduced in 2004) is not implicated, despite my expectation of a 'Google Scholar effect'.

That economic theory articles are not well cited outside of economics, and even outside of economic theory articles, will come as little surprise. Economic theory articles tend not to cover topics that are of interest outside of economics, and when they do, they are far too technical for those outside of economics to appreciate.

On the other hand, econometric methods should have a greater impact outside of the field, and it is disappointing that those papers do not seem to have an external impact. There are advances in econometrics, particularly in causal inference, that other disciplines should take advantage of. In contrast, economics increasingly involves methods drawn from applied statistics and computer science (including the machine learning and natural language processing that Galiani et al. used in this research).

Overall, the takeaway from this research is that economics is simultaneously becoming more specialised (within economic theory and econometric methods) and having a greater impact outside of the field (for applied papers). It would be interesting to see whether similar results would come from a similar analysis of articles in political science or psychology. If political science or psychology show the same pattern, that would tell us something important about how methods and results diffuse, or how they don't.

[HT: Marginal Revolution, last year]

Book review: Economics of the New Zealand Maori

In amongst my collection of books, I have assembled a number of classics, including some reasonably rare editions. One of those is Economics of the New Zealand Maori [*] by Raymond Firth. This book was originally published from Firth's PhD thesis in 1929 (the thesis was approved in 1927 at the University of London). The edition I read was the second edition (and once belonged to the Stanford University library), published in 1959 with substantial updates (you can read the first edition for free online here). Despite the age of the book, this is still an important read, as I outline below.

Firth had a background in economics, including undergraduate and graduate study at Auckland, but then shifted focus to become one of the first 'economic anthropologists'. Aside from being an astute study of what the title suggests, this book was also a demonstration of the emerging techniques of economic anthropology.

To be honest, I was expecting this book to be mildly and casually racist, but Firth is very careful in his presentation of pre-European and colonial Māori culture and economy, including for the most part avoiding the 'noble savage' portrayals that have been eviscerated in more modern 'enlightened' times. As a pākehā scholar writing in the 1920s though, there is still phrasing that made me cringe. However, this was more in the case of the gendered language, rather than the language related to Māori. Putting aside the issues of historical expression though, what the reader gets from this book is an explanation of how Māori organised themselves both socially and economically (the two being closely intertwined).

I really appreciated Firth's writing style, which has a creativity that is lacking in more contemporary academic writing. Consider this beautiful passage from the start of the second chapter:

The gannet dives, the gulls cry; no other living thing is seen. Mile after mile stretches the unbroken coast, till in the far distance the eye loses sense of form and shape in the shimmering blue haze. Inland, as covering for the Earth Mother, lies the dark forest with its mosses and dropping ferns, the virgin bush, broken only by the tiny clearings of neolithic man. At times it gives way to rolling open fern-lands, to tussock or swamp or the sage-green manuka scrub. Such is the face of Nature in the home of the ancient Maori.

The book covers a wide range of topics, from social structure, to work, 'magic', distribution, property ownership, land, and gift exchange. The last substantive chapter looks at economic aspects of cultural change, in response to contact and conflict with the settler colonial government. There is a wealth of detail on each topic, and much of the economics remains current. Of course, someone writing the book today would cover slightly different topics, or cover the topics in a slightly different way and with reference to more recent developments in economic theory (including social preferences, common resources, public choice theory, and information asymmetries). However, the core of economics as we continue to teach, including specialisation, constrained choices, and the role of incentives, are clear. Some sections even anticipate later considerations, such as moral or social incentives:

In our scheme of economic motivation we must include as powerful incentives to action the sense of communal responsibility and the desire to contribute to the well-being of the group, backed up by the strong forces of custom, habit, and tradition.

The role of social sanctions, as later developed by the work of Nobel Prize winner Elinor Ostrom, who studied how communities govern common resources, are also evident. The role of the distribution of the gains from economic activity takes a prominent role in the book, and in Firth's telling it is here where the Māori economy differs most strongly from the settler colonial economy:

However, it is clear that it was considered that all the members of the community were entitled to share in the product of any large-scale activity. It would be incorrect to picture the Maori distributive system as an idyllic kind of communism, but it is true that the manner of apportionment of goods - or food, at all events - bore direct relation to the needs of the people. Starvation or real want in one family was impossible while others in the village were abundantly supplied with food. Nor did this give the opening for idleness which one might expect. here, again, the force of public opinion stepped in, and for sheer peace the would-be slacker was obliged to defer to it and make some show of assuming his proper responsibilities. Proverbs and traditional tales also contributed to this end...

The last sentence, pointing to proverbs (whakataukī) and traditional tales, hints at another aspect of the book that I really appreciated - the links to Māori culture, and especially the retelling of stories and sayings, often accompanied by the original te reo Māori. As a beginner speaker of te reo, I always appreciate the opportunity to connect a little more with words, concepts, and sayings outside of my regular experience.

Also hinted at in the previous quote was Firth's offence at previous writers' insistence on the Māori economy as communist. In fact, he devotes a whole section of the book to debunking this idea, including:

Yet if the work of these various writers who so freely use the term communism be examined it is found that at no point do they attempt to explain what they understand by it. Apart from the absence of any definition, no consistent or detailed account of the precise operation of the communistic principle is given...

In all its varieties of meaning it retains the essential points: a common ownership of the means of production, labour contributed according to ability, and a sharing out of the fruits of industry on the basis of the needs of the members of the society... To apply the term to any vague form of group activity or group control is only to introduce needless confusion.

Shots fired! Firth carefully uncovers the private control (if not 'ownership' in the settler colonial tradition) of resources such as eel weirs, or trees for the trapping of birds, which is just one piece of evidence that demonstrates that the Māori economy was not communist. And while exchange did not occur through what an economist would think of as a market, there was nevertheless a complicated system of gift exchange within and between hapū (sub-tribes) and iwi (tribes). In terms of the latter, Firth stresses the role of generosity in distribution. From the conclusion to the book:

And so proverbs, songs, legendary tales and the stream of public opinion all combine to extol generosity in giving, open-handedness in disposing of the wealth accumulated. In the apportionment of food, in the exchange of goods the dominance of this attitude has been proven. On the whole, then, the compulsion to work, to save, and to expend is given not so much by a rational appreciation of the benefits to be received as by the desire for social recognition, through such behaviour. The entire scheme of motivation in industry is thus lifted from the biological to the social plane.

I also found Firth's writing on land tenure to be interesting. Firth wrote his thesis in the 1920s, only about 60 years after the Native Land Court (1865) was formed, and about 50 years before the Waitangi Tribunal (1975). The section on land is more in keeping with the latter:

Despite the comparatively small population in pre-European times, there was no appreciable area of land anywhere in the country which was without its owners. Districts devoid of permanent inhabitants were yet visited periodically if not for cultivation at least to obtain other food supplies... Again, the extent of ownership of land was not correlated merely with its economic productivity. The sentiment felt for it and the strength of ancestral associations, as already shown, were factors of great importance in determining ownership.

As I noted earlier, more contemporary writers would likely express this differently. For instance, 'ownership' might be too contentious a term for some Māori to use in relation to land, but the concept of a deep and abiding connection to the land, which transcends mere ownership, does come through in this book. There is also much to commend in the way that the book approaches the environmental aspects of the economy, long before sustainability became fashionable. There is also a clear and convincing discussion on the economic reasons underlying the practice of polygyny.

I really enjoyed this book. It will not be for everyone, but for those who are interested in the Māori economy this should be required reading. And those who can read it as a book written in its time, and look through some of the dated expressions to see the economic ideas that underlie the prose, will find a taonga (a treasure, or a valuable cultural artifact) that can take them a few steps along the way to understanding both the early and contemporary Māori economies.

*****

[*] New Zealand readers may wonder why I have omitted the macrons from Māori in the title of the book and from other words in the quotes. In the entire book, Firth only used a few macrons, and I quote his book verbatim (including the title) without correction.

Declining migrant rights in Europe shows Milanovic's model of migration flows and migrants' rights in action

The Financial Times reported earlier this week (paywalled):

Immigration has become more controversial since shifting from predominantly white European to predominantly non-white, non-European — mostly Asian in the UK, mostly African in France. These trends will continue: Africa’s population is forecast to jump from 1.5 billion today to 2.5 billion by 2050, while Europe’s working-age population craters...

How can politicians square the circle of needing immigrants but not wanting them? By posturing against the most visible forms of immigration (small boats on the Channel or the Med, and asylum-seekers) while quietly letting in more workers. Britain’s vote for Brexit was largely driven by anti-immigration feeling, but immigration to the UK has soared since then. Italy’s rightwing leader Giorgia Meloni makes a show of trying to process asylum-seekers in Albania — reflecting a widespread European desire to offshore asylum — while also issuing nearly a million non-EU work visas. The French parliament voted through a strict immigration law in 2023, yet in 2024 immigration jumped.

Branko Milanovic's model of migration flows and migrants' rights, which was explained in his book Capitalism, Alone (which I reviewed here) and which I expanded on in this earlier post can be used to explain these changes. Over time, Europeans have become willing to accept lower migration flows for a given amount of migration rights. In the model, this equates to a decrease in the demand for migrants (the demand in this model represents the public tolerance for migrants). This situation is shown in the diagram below. Initially, the equilibrium level of migrant rights R0 was associated with migration flows to Europe of M0. Then, as Europeans' demand for migrants decreases, the 'demand curve' shifts to the left, from D0 to D1. The equilibrium now occurs where the new curve D1 intersects with the supply curve S0, with lower migration flows (M1) and lower migrant rights (R1).

As the Financial Times article notes, migrant rights are eroding:

The new trend, as seen for instance in the UK, is to give immigrants time-limited visas for specific job sectors, reduce their right to bring family members, and make them wait longer — decades, in some cases — before they can get permanent settlement. In France, the far-right Rassemblement National party, the likely next government, wants to scrap birthright citizenship, meaning that people could spend their lives in the country while forever remaining second-class outsiders.

Milanovic's model helps us to explain how changes in Europeans' preferences for migrants translate into both lower migration flows and lower migrant rights. To improve migrants' rights, the process would likely have to happen in reverse, with Europeans returning to a more welcoming state.

Read more:

• Milanovic's model of migration flows and migrants' rights, expanded

This week in research #106

Here's what caught my eye in research over the past week:

• Steenbrink and Skali (open access) look at the relationship between wealth inequality and economic growth, finding that a one standard deviation increase in the wealth Gini coefficient within countries is associated with a 0.34 percentage points decline in growth rates
• Liu and Wang (with ungated earlier version here) use website traffic and Google Trends data to investigate who is using generative AI, and find that users skew young, highly educated, and male, particularly for video generation tools, and that country-level adoption intensity is strongly correlated with the share of youth population, digital infrastructure, English fluency, foreign direct investment inflows, services’ share of GDP, and human capital
• Baumann and Svec examine peer effects in the real-time strategy computer game StarCraft 2 and find that having higher quality teammates is associated with a rise in the player’s ratings over the next two weeks and that the impact grows over time
• Cook examines the effect of home stadium attendance on a variety of match outcomes in the NFL, using the COVID-affected 2020 season as an instrumental variable, and finds that home stadium fans influence the outcomes of NFL games in favour of the home team, by directly affecting the away team's performance
• Risse (open access) finds that adding unpaid work and care to paid work, and adjusting for the undervaluation of female-concentrated jobs and the male wage premium, sees women’s share of total labour input expand from 36.8 per cent to 50.5 per cent in Australia
• Melnikov, Schmidt-Padilla, and Sviatschi (open access) find that individuals in gang-controlled neighbourhoods in El Salvador have less material well-being, income, and education than individuals living only 50 meters away but outside of gang territory
• Rubenstein and Nesbit (open access) find that the new Guardian Caps in the NFL do not significantly lower concussion rates, and in fact increase concussions in the position groups where using Guardian Caps was mandated

And the latest paper from my own research (jointly with my former PhD student Muhammad Irfan and my colleague Waqar Akram):

• Our new article (open access) in the Journal of Energy and Development looks at the adoption of household solar energy in Pakistan, and shows that, as income and education increase, grid-sourced electricity will remain the preferred source of household energy, but since grid-source electricity is unavailable in rural areas, solar energy would need more government support in those areas

Ask not what economics can do for sports; ask what sports can do for economics

Regular readers of this blog will know that I enjoy blogging about research that uses a sports setting to illustrate economic concepts (except when the research is terrible). Sport makes for an interesting setting for testing economic theories. The rules are known. The incentives are usually clear. The outcomes are usually unambiguous. Other real-world settings don't provide the same clarity. This matters because sports can tell us something about real-world behaviour that lab experiments can’t. And essentially, that is one of the points that Ignacio Palacios-Huerta (London School of Economics) makes in this new review of the literature, published in the Journal of Economic Literature (ungated earlier version here).

As a researcher, Palacios-Huerta has frequently exploited sports data (and has a book using data from football, Beautiful Game Theory, which makes some of the same points as this review, but with a narrower focus on football, or soccer). In this review, Palacios-Huerta argues that:

...many sports settings are in fact natural experiments that “happen to occur” and provide evidence that is as direct and convincing as controlled experiments... And yet other settings offer such a clean environment for identification that they seem designed as a perfect vacuum for measuring theoretical postulates. The precise knowledge of participants’ goals and rules and the clear observability of strategies, incentives, actions, and consequences in these settings is indeed rare in other field settings.

Mainstream economics mostly ignores sports as a source of data and a test of theories. Palacios-Huerta notes that this is limiting scientific progress in economics:

A reluctance to view sports settings from this perspective may reflect a deep misunderstanding of the virtues of sports data. And this reluctance has discouraged the study of these settings and slowed down the production of knowledge in economics and other social sciences.

The review then provides a wide review of specific examples where sports data has informed tests of economic theories. The range of theories is too broad to note in full here, but encompasses things like game theory, risk and uncertainty, behavioural economics, market design, labour economics, discrimination, technological change, and lots of examples of the roles of incentives. The variety of sports that provide the data is also vast, including football (of all kinds), golf, chess, basketball, baseball, motor racing, and even biathlon and gymnastics. Surprisingly, there is no mention of cycling, athletics, or swimming. My only disappointment is that Palacios-Huerta doesn't draw on e-sports (although they do make an appearance in the appendix to the paper). He could also have given a nod to game shows, which provide similar benefits to sports (and are also a favourite topic for me to blog about).

If you're looking for data to test a particular theory, sports might help. But sports can offer more than just data: they can generate new insights into theory itself. As Palacios-Huerta notes in the conclusion to the review:

...sports settings offer more than unparalleled opportunities for measurement, verification, and falsification; they are unique from the perspective of a key component of science that is largely ignored: discovery... Ironically, one of the main benefits of the excessive highbrowism that sports have endured is that they remain an important and largely unexplored setting for discovering new phenomena and hypotheses.

The opportunities that sports offers for economists are broad, and sports data are underutilised. And, economics research that uses sports as a setting can have broader appeal as well. Hopefully, economists are listening.

Read more:

• The increasing role of economics in sports, and teaching sports economics

Grade inflation at New Zealand universities, and what can be done about it

Grade inflation at New Zealand universities has been in the news recently. This is a delayed reaction to this report from the New Zealand Initiative released back in August, authored by James Kierstead. He collected data on grade distributions from all eight New Zealand universities (via Official Information Act requests), and looks at how those distributions have changed over time. The results are a clear demonstration of grade inflation, and most clearly demonstrated in Figure 2.1 from the report:

Over the period from the mid-2000s to 2024, the proportion of New Zealand university students receiving a grade in the A range has increased at every New Zealand university, and by more than ten percentage points overall. Kierstead notes that:

Overall, the median proportion of A-grades grew by 13 percentage points, from 22% to 35%... The largest increases occurred at Lincoln, where the proportion of As grew by 24 percentage points between 2010 and 2024 (from 15% to 39%), more than doubling, and Massey, where they grew by 17 percentage points (from 19% to 36%) from 2006 to 2023.

A similar pattern of increases, although not as striking, is seen for pass rates, which in 2024 were above 90 percent at every university except Auckland. The results are also apparent across different disciplines, as shown in Figure 2.4 from the report:

Of course, this sort of grade inflation is common across other countries as well, and Kierstead provides a comparison that shows that New Zealand grade inflation is not dissimilar from grade inflation in the US, UK, Australia, and Canada.

Kierstead then turns his attention to why there has been grade inflation. He first dismisses some possible explanations such as better incoming students (NCEA results have not improved, although even if they had that might be due to grade inflation as well), more female students (the proportion of female students has been flat over the past ten years, while grades have continued to increase), better funding (bwahahahaha - in fact, funding per student has declined in real terms since 2019, while grades have continued to increase), and student-staff ratios (which have declined over time, but the student-academic ratio, which is the one that should matter most, has barely changed).

So, what has caused grade inflation? Kierstead describes it as a collective action problem, akin to the tragedy of the commons first described by Garret Hardin in 1968:

It is our contention that grade inflation is the product of a dynamic that is not dissimilar to the tragedy of the commons. Just like Hardin’s villagers, academics pursue a good (in this case high student numbers) in a rational way (in this case by awarding more high grades). And just as with Hardin’s villagers, negative consequences ensue, with a common resource (sound grading) being depleted, to the cost of every individual academic as well as others...

In the grade inflation game, the good that academics want to maximize is student numbers. Individual academics, on the whole, want to have as many students in their courses as possible. This suggests that they are popular teachers and can help get them promoted (and hence gain more money and prestige). It can also help make sure the courses they want to teach stay on the menu.

I like this general framing of the problem, where 'sound grading' is a common resource - a good that is rival and non-excludable. However, I would change it slightly, by thinking about the common resource as being A grades generally, which are depleted when the credibility of those grades reduces. In my slightly different framing, awarding A grades is rival in the sense that one person awarding more A grades reduces the credibility of A grades awarded by others. Awarding A grades is non-excludable in the sense that if anyone can award A grades, everyone can award A grades (while it is possible to prevent academics from awarding A grades, universities would probably prefer not to do so because that would reduce student satisfaction). So, while the social incentive for all academics collectively is to reduce the award of A grades to keep the credibility of those grades high, the private incentive for each academic individually is to increase the proportion of A grades awarded, leading to fame and fortune (or, more likely, leading to fewer awkward conversations with their Head of School as to why their grade distribution is too low, as well as better student evaluations - see here and here, for example). Essentially then, the incentives are for academics to inflate grades. The universities have few incentives to act to reduce grade inflation, since higher grades increase student satisfaction and lead to greater enrolments.

However, there is a problem. As Kierstead notes, grade inflation is well-termed because its effects are similar to the inflation that economists are more familiar with:

If universities hand out more and more As in a way that isn’t justified by student performance, the value of an A will go down. The same job opportunities will ‘cost’ more As as As flood the market. Students who worked hard will see the value of their As decrease over time, just as workers in the economy see their savings decrease in value due to monetary inflation.

So, what to do? Kierstead offers a few solutions in the report, including moderation of grades, reporting grades differently on transcripts, calculating grades differently, making post-hoc adjustments to grade point averages, having national standardised exams by discipline, changing the way that universities are funded to reduce the incentive to inflate grades, changing the culture of academics, and giving out prizes for 'sound grading'. I'm not going to dig into those different solutions, because sometimes the simplest one is the best one. With that in mind, I pick this:

Perhaps the simplest addition that could be made to student transcripts alongside letter grades is the rank that students achieved out of the total number of students on the course. So a student’s transcript might read, for example, ‘Classics 106: Ancient Civilizations: A- (27th of 252).’...

Adding ranking information restores some of the signalling value of grades without needing to reverse grade inflation itself. To see why, consider an example. If an employer has the transcripts of two students, one of whom got an A- grade in econometrics and ranked 17th out of 22 students, while the other student got a B grade and ranked 3rd out of 29 students, it's pretty clear that the grade might not be capturing the full picture of the students' relative merit. Kierstead worries about this simple solution because:

A limitation of rank-ordering is that it might suggest that students who achieved only a lowly ranking had performed badly, whereas they might well have performed very well in an especially difficult course.

Possibly, but the key point is not how well students did in the course, but how well they did relative to the other students in the class, which is exactly what the ranking provides. The benefit of this approach is that providing a ranking alongside the grade would reduce the incentives for students to cherry pick easy papers that award high grades, because a high grade on its own would not necessarily lead to a good ranking within the class.

Of course, there are potential problems with the simple solution. One such problem is that comparisons across different cohorts of students might not be fair. Taking the example of the two students I gave earlier, perhaps the student who got an A- grade and ranked 17/22 completed the paper in a cohort that was particularly smart, while the student who got a B grade and ranked 3/29 completed the paper in a cohort that was less smart. In that case, the grade without the ranking might be a better measure.

Kierstead's more complex solutions don't really deal well with the problem of between-cohort comparisons, and suffer from being more complicated for non-specialists to understand. A simple ranking, or a percentile ranking, is relatively easy for HR managers to interpret. Having said that, the between-cohort comparisons issue might not be too much of a problem in any case. My experience though, is for classes of a sufficiently large size (30 or more), the grade distributions do not differ materially (and if they do, it is usually because of the teaching or the assessment, not the students).

I can see some incentive issues though. Would students start to choose papers that they suspect that many weak students complete? Good students might anticipate that this would lead to a higher grade and a better ranking, which will look better on their transcript. On the other hand, is that really any worse than what students are doing now, if they choose papers that give out easy grades?

There are also potential issues with stigmatising students who end up near the bottom of a large class (how dispiriting would it be to have your transcript say you got a grade of E, and ranked 317th out of 319 students?). Of course, that could be solved to some extent by only providing ranking information for students with passing grades. And consideration would also be needed for how to deal with very small classes (is a ranking of 4th out of 5 students meaningful?).

Grade inflation is clearly a problem. It's not just nostalgia to say that an A grade is not what it used to be. Grade inflation has real consequences for employers, because the signalling value of high grades is reduced (see here for more on signalling in education). This means that there are also real consequences for high-quality students, who find it more difficult to differentiate themselves from average students. Solving this problem shouldn't involve government intervention to change university funding formulas, or trying to change academic culture. It shouldn't involve complicated statistical manipulations of grades. It really could be as simple as reporting students' within-class ranking on their academic transcripts.

The question now is whether any university would take it on themselves to do so. The credibility of university grades depends on it.

[HT: Josh McNamara, earlier in the year]

Read more:

• Teaching evaluations and grade inflation
• Grade inflation is harming students' learning
• A few papers on grade inflation at universities
• Grade inflation and college completion in the U.S.
• More on teaching evaluations and grade inflation

Online and blended learning lead to similar outcomes on average, at lower cost but lower student satisfaction

It's been a while since I've written about online or blended learning, which may seem surprising given the ample opportunities for us to learn about online learning during the pandemic. Perhaps I'm still dealing with the trauma of that, or perhaps I have just pivoted more to understanding the emerging role of AI in education. Nevertheless, I recently dipped my toes back into the research on online and blended learning, reading this 2020 article by Igor Chirikov (University of California, Berkeley) and co-authors, published in the journal Science Advances (open access).

Chirikov et al. evaluate a large multisite randomised controlled trial of online and blended learning in engineering, across three universities in Russia. As they explain:

In the 2017–2018 academic year, we selected two required semester-long STEM courses [Engineering Mechanics (EM) and Construction Materials Technology (CMT)] at three participating, resource-constrained higher education institutions in Russia. These courses were available in-person at the student’s home institution and alternatively online through OpenEdu. We randomly assigned students to one of three conditions: (i) taking the course in-person with lectures and discussion groups with the instructor who usually teaches the course at the university, (ii) taking the same course in the blended format with online lectures and in-person discussion groups with the same instructor as in the in-person modality, and (iii) taking the course fully online.

The course content (learning outcomes, course topics, required literature, and assignments) was identical for all students.

Their sample is made up of 325 second-year university students, with 101 randomly assigned to in-person, 100 to blended, and 124 to online. All students then completed the same final examination. Looking at student performance, Chirikov et al. find:

...minimal evidence that final exam scores differ by condition (F = 0.26, P = 0.77)... The average assessment score varied significantly by condition (F = 3.24, P = 0.039): Students under the in-person and blended conditions have similar average assessment scores (t = 0.26, P = 0.80), but those under the online condition scored 7.2 percentage points higher (t = 2.52, P = 0.012). This effect is likely an artifact of the more lenient assessment submission policy for online students, who were permitted three attempts on the weekly assignments.

The lack of a difference in student performance on average across different learning modes is a common feature of the literature (see the links at the end of this post). It would have been interesting if Chirikov et al. had undertaken a heterogeneity analysis to see whether online and blended modes advantage the more able and engaged students, while disadvantaging the less able and engaged students (also a feature of the literature on online and blended learning). The general result that online and blended learning provides benefits for top students but harms weaker ones is a point I’ve discussed many times before (see the links below for more).

Chirikov et al. then look at student satisfaction, and despite claiming that "we find minimal evidence that student satisfaction differs by condition", Table 3 in the paper does show that students in the online mode report a statistically significant five percentage points lower satisfaction than in-person students, while students in the blended mode report lower satisfaction (by about 2-2.5 percentage points) than in-person students, although the latter difference was not statistically significant.

Finally, Chirikov et al. evaluate the effect on the cost of education, finding that:

Compared to the instructor compensation cost of in-person instruction, blended instruction lowers the per-student cost by 19.2% for EM and 15.4% for CMT; online instruction lowers it by 80.9% for EM and 79.1% for CMT...

These cost savings can fund increases in STEM enrollment with the same state funding. Conservatively assuming that all other costs per student besides instructor compensation at each university remain constant, resource-constrained universities could teach 3.4% more students in EM and 2.5% more students in CMT if they adopted blended instruction. If universities relied on online instruction, then they could teach 18.2% more students in EM and 15.0% more students in CMT.

I don't think it will come as a surprise to anyone that online and blended learning are more cost-effective. There is little doubt that it has factored into some of the push towards online and blended learning across higher education over time.

Given that, in this study, both online and blended learning lead to similar outcomes on average, one might be tempted to suggest that they are good value for money from the university’s or funder's perspective. For cash-strapped institutions (or governments), the temptation to expand online provision on the back of such numbers is obvious. However, we should be cautious about drawing that conclusion. The lower student satisfaction in the blended and (especially) online modes should be a worry (at least to those who care about student satisfaction). And, as alluded to earlier, the average student performance can hide important heterogeneity between more engaged and less engaged students.

The real question here isn’t whether online and blended learning can be as effective on average, but whether we are comfortable trading lower satisfaction and potential for harms to less engaged students for lower cost of delivery and higher enrolments.

Read more:

• Online vs. blended vs. traditional classes
• Flipped classrooms work well for top students in economics
• Flipped classrooms are still best only for the top students
• Online classes lower student grades and completion
• Online classes may also make better students worse off
• Meta-analytic results support some positive effects of using videos on student learning
• New review evidence on recorded lectures in mathematics and student achievement
• Meta-analytic results may provide some support for flipping the classroom
• The value of an in-person university education
• The pandemic may have revealed all we need to know about online learning
• Live streamed video lectures and student achievement
• Low-performing students, online teaching, and self-selection
• A slightly more optimistic take on online learning during the pandemic
• Research-based instructional strategies and flipped classrooms in economics
• Using microcommitments and social accountability to improve online and hybrid learning
• What pandemic schooling tells us about online learning
• We should be careful not to conflate the effect of online revision with the effect of online teaching
• The impact of remote learning in Brazilian high schools

This Kansas City Chiefs conspiracy theory article is a mess

I have to admit to experiencing a non-trivial amount of schadenfreude this year, as the Kansas City Chiefs find themselves with a losing record in December for the first time in a decade. My mild animosity towards the Chiefs is based entirely on their supreme performance over that decade. After they've had a few losing seasons, I won't care anymore (which is how I feel about the Patriots right about now). However, there are plenty of people who have griped about the Chiefs, and claimed that the Chiefs receive favourable referee calls.

I'd label that a conspiracy theory, but it has apparently caught the attention of researchers. This recent article by Spencer Barnes (University of Texas at El Paso), Ted Dischman (an independent researcher), and Brandon Mendez (University of South Carolina), published in the journal Financial Review (sorry, I don't see an ungated version online), explicitly tests whether the Kansas City Chiefs receive favourable referee calls. Specifically, Barnes et al.:

...compare penalty calls benefiting the Mahomes-era Kansas City Chiefs (from 2018 to 2023) and the Brady-era New England Patriots (2015–2019) across the regular and postseason...

Barnes et al. argue that:

...financial pressures, particularly those related to TV revenue (the primary source of revenue for the NFL), serve as the underlying mechanism.

In other words, Barnes et al. claim that the NFL has a strong financial incentive to bias officiating in favour of the 2018-2023 Kansas City Chiefs, to a greater extent than any bias in favour of the 2015-2019 New England Patriots. As we’ll see, the empirical strategy is poorly chosen, parts of the results are misinterpreted, and the proposed TV-revenue mechanism is implausible. All up, you shouldn't believe this paper's results.

What did they do? Barnes et al. use play-by-play data covering the 2015 to 2023 seasons. They restrict their attention to defensive penalties only, which gives them a sample of 13,136 penalties across 2435 games. They apply a fairly simple linear regression model to the data:

Here we find the first problem with their analysis. If you want to show that the Mahomes-era Kansas City Chiefs benefited from more defensive penalties than other teams, you should be running a difference-in-differences analysis. Essentially, you compare the difference between the Chiefs and other teams, between the period before and the period after Patrick Mahomes started playing. In other words, you should test whether the Chiefs’ advantage in penalties grows after Mahomes started playing, compared with their earlier advantage and with other teams over the same period. Barnes et al. simply test for a level difference between the Chiefs and other teams during that time (using the 'Dynasty' variable), but fail to account for whether the Chiefs might already benefit from more defensive penalties before Mahomes became the starting quarterback (in 2018). Indeed, Figure 1 in the paper shows that the Chiefs did benefit from more defensive penalties per game before 2018:

That difference prior to 2018 should be controlled for. Having said that, the difference from the rest of the NFL teams looks bigger from 2018 onwards (but mostly concentrated in 2018-19, and in 2023), so if they had used the more correct difference-in-differences model (or, when comparing regular and post-season, a triple-differences model), they might still have found a statistically significant effect.

There is a further, albeit more minor, issue with the analysis. Barnes et al. control for 'defensive team fixed effects', which they argue controls "for differences in how opposing teams play defense and how frequently they are penalized". However, teams change the way they play defence, particularly when the defensive coordinator changes. So really, they should have used defensive-team-by-season fixed effects there, which would allow the way a team plays (and gets penalised) to vary from season to season, and control for that.

Barnes et al. look at the effect on several outcome variables:

Our primary dependent variables capture different dimensions of officiating decisions. The first is Penalty Yards, which measures the total yards gained or lost due to penalty calls. If the NFL or its officials favor a particular team, we expect them to benefit from potentially more penalty yards assessed against their opponents. The second variable, First Down, is a binary indicator that takes a value of 1 if a penalty call results in an automatic first down. Because first downs have a direct impact on a team’s ability to sustain drives and score points, this measure captures whether penalties disproportionately help a team advance the ball. The third variable, Subjective, is a binary indicator equal to 1 if the defensive penalty falls into a category requiring referee discretion...

The 'Subjective' variable is described in the appendix to the paper, and appears to be far too inclusive since it includes penalties like 'Face Mask' and 'Horse Collar Tackle' that seem to me not to be particularly subjective (and those two categories alone made up 6 percent of all penalties, and a much higher proportion of the 'subjective' penalties).

Putting aside the issues with the analysis for a moment, Barnes et al. find that:

...penalties against Kansas City during the regular season result in 2.02 fewer yards (𝑝 < 0.01), are 8 percentage points less likely to have a penalty call that results in a first down (𝑝 < 0.01), and are 7 percentage points less likely to have subjective penalties (𝑝 < 0.05) compared to the rest of the NFL. This pattern is decisively reversed in postseason contests, where penalties against the Chiefs offense yield 2.36 more yards (𝑝 < 0.05), are 23 percentage points more likely to have a penalty call that results in a first down (𝑝 < 0.01), and are 28 percentage points more likely to have subjective calls (𝑝 < 0.01) compared to the rest of the NFL in the playoffs.

Barnes et al. have explained this incorrectly. Notice their wording suggests the penalties are called on Kansas City (i.e. hurting the Chiefs). Their analysis actually shows that penalties against Kansas City Chiefs' opponents result in 2.02 fewer yards during the regular season, and penalties against Kansas City Chiefs' opponents (not the Chiefs offense) yield 2.36 more yards in the postseason. At least, that is according to the notes to their Table 3, which says:

The dependent variable in Columns (1) and (4) is the realized yardage for the offensive team resulting from a penalty on the defensive team... The independent variable of interest, Kansas City Chiefs, is a binary indicator variable that equals 1 if the offensive team is the Kansas City Chiefs and 0 otherwise.

So, the correct way of interpreting those results is penalties against the opposing defence, not penalties against Kansas City. Barnes et al. then turn to applying the same analysis to the 2015-2019 New England Patriots, and find effects that are mostly statistically insignificant (and small). For other teams that might arguably be called a 'dynasty' (for a sufficiently low bar for what constitutes a dynasty, Barnes et al. find no evidence of differences in defensive penalty calls. That sample includes the Philadelphia Eagles (2017-2023), the Los Angeles Rams (2018-2023), and the San Francisco 49ers (2019-2023).

At this point, the problem with the mechanism starts to become clear. Barnes et al. start to look at TV viewership, and argue that:

If certain teams, particularly those associated with high-profile players, systematically attract larger audiences, then maintaining the success or visibility of those teams may align with the league’s broader financial interests.

If the NFL wanted to attract a larger audience, and aimed to do so by biasing officiating in favour of a particular team, why on earth would they choose a small market team like Kansas City? Surely they would want to boost a large-market team? According to this ranking, Kansas City is only the 35th-largest sports media market in the US. Now, Patrick Mahomes is a star quarterback (he was the 10th overall pick in the 2016 NFL draft), so maybe it's the combination of star quarterback and media market that matters. However, Tom Brady was also a star quarterback, and Boston is the 10th-largest sports media market. So, why weren't the Patriots getting favourable calls in 2015-2019? If, as Barnes et al. seem to argue, the NFL was going through some particular challenges in 2016, then Kansas City is still not the obvious choice for biased officiating. They should have favoured the LA Rams (in the second-largest sports media market, with star quarterback Jared Goff, the first overall pick in the 2016 NFL draft).

Barnes' et al.'s argument falls apart. Their TV viewership analysis does show that:

...the Chiefs’ emergence as a marquee team coincided with a material increase in viewership interest, consistent with the broader financial incentives we hypothesize.

However, that analysis also has issues, because they don't control for the win/loss record of the teams in each game (and winning teams likely attract more TV viewers). And, all it really tells you is that Patrick Mahomes attracts a big TV audience. He is a good player. That's what they do. Higher ratings for teams with star players is not evidence that referees are biased. As noted above, if the NFL thought that way, they should have preferred biasing the officiating towards the LA Rams instead, and Barnes et al.'s analysis shows that didn't happen.

As a final point, there is a real risk that the analysis in this paper gets causality backwards. Did the Chiefs get favourable referee calls because they are a dynasty, or did they become a dynasty because they received favourable referee calls at key moments? Barnes et al. never consider the possibility of reverse causality. Overall, the paper does much more to flatter an existing conspiracy theory than to seriously test it. Even if we take their estimates at face value, nothing in the paper convincingly links referee calls to incentives to increase NFL TV viewership.

[HT: Marginal Revolution]

This week in research #105

Here's what caught my eye in research over the past week:

• Gillespie et al. (with ungated earlier version here) find evidence of landlord exit from the rental market, specifically after rent controls were tightened in 2021 in Ireland, meaning that rent controls are associated with more sale listings and fewer rental listings/registrations
• Pagani and Pica (open access) find that exposure to a higher share of same-gender math high achievers is related to better academic performance among Italian primary school children, for both boys and girls, three years later
• Dutta, Gandhi, and Green (open access) find, using data from India, that relaxing rent control leads to higher rents and decreases rural-urban migration, while easing eviction laws increases the conversion of rental units into owner-occupied housing and increases the prevalence of 'marriage migrants'
• Couture and Smit find no evidence that Federal Open Market Committee officials in the US select securities that earn abnormal returns
• Bergvall et al. (open access) find, using Swedish data, that find that following the start of their PhD studies, psychiatric medication use among PhD students increases substantially, continuing throughout their studies to the point that by the fifth year medication use has increased by 40 percent compared to pre-PhD levels (more reason to worry about the mental health of PhD students)
• Bagues and Villa (open access) find that, after Spanish regions increased the minimum legal drinking age from 16 to 18 years, alcohol consumption among adolescents aged 14-17 decreased by 7 to 17 percent and exam performance improved by 4 percent of a standard deviation
• Fan, Tang, and Zhang find, using data on university relocations in China in the 1950s, that there were substantial effects on total employment, firm numbers, and productivity in industries technologically related to the relocated departments
• Chikish and Humphreys find that surgical repair of UCL injuries extends post-injury MLB pitcher careers by roughly 1.3 seasons relative to matched uninjured pitchers, and that post-injury and treatment pitcher performance improves by roughly 8 percent
• Chegere et al. (open access) conduct an experiment investigating how regular sports bettors in urban Tanzania value sports bets and form expectations about winning probabilities and find that people assign higher certainty equivalents and winning probabilities to sports bets than to urn-and-balls lotteries with identical odds, even though, in fact, they are not more likely to win
• Seak et al. (with ungated earlier version here) find that experimental choices by both humans and monkeys violated the independence axiom across a broad range of reward probabilities (both monkeys and humans are not purely rational decision-makers)

Do men and women pitch science proposals differently, and does it matter for funding outcomes?

If male academics and female academics write academic papers and grant proposals differently, does that lead to different outcomes by gender? Past studies have worried about whether grant funding decisions are affected by gender bias (see here, for example), and differences in writing style may contribute to that. However, the article I discussed in this post from earlier this year concluded that there was little evidence of bias in grant funding, at least since 2000 in the US.

Nevertheless, I thought it would be interesting to read this 2020 article by Julian Kolev (Southern Methodist University), Yuly Fuentes-Medel, and Fiona Murray (both MIT), published in the journal AEA Papers and Proceedings (ungated here), because it not only looks at the grant funding decisions, but also at writing style. Kolev et al. focus on grant applications submitted to the Bill and Melinda Gates Foundation and the National Institutes of Health (NIH) over the period from 2008 to 2016. The sample includes 6931 Gates Foundation applications and 12,589 NIH applications.

Kolev et al. first subject the applications to textual analysis to the abstract of each application, evaluating the positivity of the text (and the extent to which the word "novel" is used), the readability (using the Flesch reading ease score), concreteness of the language (as opposed to abstractness), and three measures of how narrow or broad the abstract is. In this textual analysis, they find that:

...female applicants are less likely to present their research using positive vocabulary, they are more likely to write with high readability, and they prefer concrete language. Moving to our final three measures, we find an interesting dichotomy: even as female applicants use fewer broad words and more narrow words in their abstracts, we find that their research is characterized by lower MeSH concentrations, meaning that they cover a wider range of medical subjects in their work, at least within the NIH sample. Effect sizes are relatively small: the impact of gender ranges from approximately 0.04 to 0.08 standard deviations for our significant effects.

So, there are small but statistically significant differences in writing style between male academics and female academics in these funding proposal abstracts. Does that translate into differences in outcome? Kolev et al. test for whether the measures of writing style correlate with funding outcome, while controlling for:

...calendar time and application topic fixed effects, controls for total word count and the count of relevant words for dictionary-based metrics, and applicant publication history and gender.

In this analysis, Kolev et al. find that:

For Gates applicants, high levels of concreteness tend to improve the odds of funding; by contrast, at the NIH, we find a strong positive impact for MeSH concentration and marginal effects for both broad and narrow words.

So, the evidence is weak that writing style matters, or that writing style differences between the genders affect the success of funding applications. However, not so fast. There is a key problem with this analysis. If you read the quote above about the control variables in this second analysis, you may note that they control for gender. That might sound sensible, but if you're wanting to evaluate whether writing style differences between the genders affect funding outcomes, you don't want to control for both writing style and gender. What Kolev et al. have actually tested is whether writing style differences within each gender affect funding application outcomes, finding that they don't. As an example, their analysis doesn't answer the question of whether readability differences between male and female academics affect funding outcomes, it answers the question of whether readability differences matter overall (which it appears they don't), controlling for the average difference in funding outcomes between men and women. Those are quite different questions.

In other words, we probably want to know whether style mediates the effect of gender on funding outcomes, but this analysis doesn't do that. Instead, they should either run the analysis with gender as the main explanatory variable, then add the style variables and see if the coefficient on the gender variable shrinks, or run the analysis with interactions between gender and the style variables.

The results are, on the one hand, surprising. Quality of writing should matter. Stylistic differences should matter less. However, the quality of the proposed research should matter even more than the quality or style of writing. And this study wasn't even evaluating the quality or style of all of the writing (or the quality of the proposed research), only the style of writing in the abstract for the proposal. That, along with the issue with the second analysis above, make this paper of limited use for understanding whether there is a gender difference in funding outcomes (and, if there is, whether writing style differences contribute to the difference). The difference in writing style is an interesting result in itself, but we need to know more.

Read more:

• The gender gap in economics
• More on the gender gap in economics
• CORE on 'missing women in economics'
• Female representation in economics
• Justin Wolfers on why we should care about the gender gap in economics
• Why we should care about the gender gap in economics
• Could closing the gender gap in economics be as simple as providing students with information?
• A sobering report on the culture in the economics profession
• The gender gap in reviewing and editing for top economics journals
• The gender gap in U.S. economics education
• UWE and the gender gap in economics (results pending?)
• Three papers on the gender gap in economics seminars
• Disappointing results on simple information interventions to close the gender gap in economics
• Gender differences in fields of specialisation within economics
• Gendered citations of papers in the 'top five' economics journals
• The positive effect of mentoring for female economists
• What machine learning is telling us about the gender dynamics in economics seminars
• There's something about Wellesley
• The gender gaps in academia may not arise entirely from gender biases

People care about whether their data are shared, but not so much where their data are stored

There has been a substantial policy movement in favour of the localisation of data storage over the past decade (for example, see here). Policymakers often justify data localisation policies by appealing to consumers' supposed preference for having their data stored locally. In particular, they refer to privacy concerns, lack of trust in data handling practices in other countries, and preference for supporting local data storage firms. However, the evidence that consumers have strong preferences for data localisation is very thin. In fact, this new article by Jeffrey Prince (Indiana University) and Scott Wallsten (Technology Policy Institute), published in the journal Information Economics and Policy (ungated earlier version here), may represent the first attempt to really evaluate consumers preferences for data storage.

Prince and Wallsten use a discrete choice survey to evaluate preferences for localisation for different types of data. Specifically:

We constructed five different survey structures, one each centered on the respondent’s smartphone, financial institution, healthcare app, smart home device, and social media. The data types we consider include home address, phone number, income, financial activity, health status and activity, biometrics, music preferences, location, networks, and communications. Across the five survey structures and range of data types, we measure the relative value of full privacy (no data sharing) versus sharing only domestically (localization), sharing domestically and internationally (no localization), and sharing domestically and internationally excluding China and Russia (no localization but with limits). We administered each of these five different surveys across seven different countries: the United States, the United Kingdom, South Korea, Japan, Italy, India, and France

Their sample, drawn from Dynata's online panel, is 11,375 respondents, with 325 completed surveys for each of the five survey structures, for each of the seven countries. Each respondent was shown ten different discrete choice questions. In each question, respondents would have been shown hypothetical alternative scenarios about how their data could be stored and shared, and had to pick their preferred alternative. However, the article doesn't make clear how many alternatives the respondent was choosing from in each choice task, nor whether they simply chose the best of the alternatives, or provided a full ranking of all of the alternatives. Those are issues that are consequential for the analysis, but probably don't bias the results in any way.

In terms of data localisation, Prince and Wallsten distinguish between data not being shared at all, and data being stored and shared domestically only, internationally, or internationally while excluding China and Russia. The latter is included because consumers may be more concerned about their data being stored in China or Russia than being stored in other countries. First, Prince and Wallsten find that:

...virtually all of our parameter estimates are highly significant. As these are estimates of (dis) utility from sharing data in one of three ways (domestically only, internationally, internationally except China and Russia) versus not sharing, the consistent, negative and statistically significant estimates imply that respondents across all of our countries are averse to sharing their data.

In other words, people really don't like their data being shared, regardless of how or where it would be shared. However, in terms of data localisation, Prince and Wallsten find that:

...it is evident that there are a just handful of data types for which we find any notable data localization premium: bank balance, facial recognition, home address, and phone number, all with multiple instances, and voiceprint, with one instance.

Interpreting these results, Prince and Wallsten note that:

...the data types for which we find a data localization premium are also the data types for which citizens find the most value in having no sharing of any kind... citizens across our seven countries, by and large, place little to no value in data localization requirements, despite placing value on full privacy for these data (i.e., no domestic or international sharing)...

In addition, there are no differences between sharing internationally, and sharing internationally while excluding China and Russia. If anything, there is some weak evidence that respondents in South Korea and Japan preferred to have their data shared with China and Russia. That’s striking given how often policymakers highlight the dangers of data flowing to China and Russia. Prince and Wallsten conclude that:

Our findings have several implications. First, they suggest that the use of privacy concerns as motivation for data localization laws may be overstated, although there may be some gross welfare gains for some types of data. Our findings also indicate that if international sharing is allowed, restricting prominent authoritarian countries such as China and Russia appears to have little impact on consumer value, at least for a number of highly populated countries...

...our findings do provide a counterweight to any claim that citizens find value from imposing constraints on international data sharing.

It may still be worthwhile for policymakers to insist on data localisation. Of course, this is just one study (albeit the first study) using survey data from an online panel, so we should be cautious about overgeneralising. Nevertheless, based on this study, the argument that data localisation reflects consumers' preferences for data storage does not hold up to scrutiny. If they want to keep pushing data localisation, policymakers will need to lean on geopolitical or protectionist arguments instead.