The ethnographic atlas isn't 'tabulated nonsense'

I've seen a number of papers over the years that have made use of data from George Murdock's Ethnographic Atlas (see here for a gated summary, or here for the data), often as an instrument for some other variable (as one example, see here). The Atlas summarises ethnographic data from over 1200 pre-industrial societies, including a variety of characteristics such as political organisation, social organisation, norms, and agricultural practices. A search on Google Scholar reveals that it has been cited over 6700 times, so it is widely used.

Given widespread use of the Ethnographic Atlas , I was very interested when I saw the title of this new article by Duman Bahrami-Rad, Anke Becker, and Joseph Henrich (all Harvard University), published in the journal Economics Letters (ungated earlier version here): "Tabulated nonsense? Testing the validity of the Ethnographic Atlas". It turns out that I didn't need to worry too much, and nor should researchers using data from the Ethnographic Atlas. Bahrami-Rad et al. compare data from the Ethnographic Atlas with comparable variables from more recent Demographic and Health Surveys for the same ethnic groups. They find:

...positive associations between the historical information reported by ethnographers and the contemporary information reported by a large number of individuals. Importantly, the associations between historical ethnicity-level measures and contemporary self-reported data do not only hold for dimensions that would have been easy to observe for an ethnographer, such as how much a society relies on agriculture, or whether marriages are polygynous. Rather, they also hold for dimensions that are more concealed, such as how long couples abstain after birth, or whether people prefer sons.

Clearly, no cause for concern. The title of the paper is clickbait (the quote "tabulated nonsense" is attributed to the British anthropologist Sir Edmund Leach), and clearly effective, since it got me to read the paper.

The PBRF has served its purpose and it's time for a re-think

New Zealand's Performance Based Research Fund (PBRF) has been going through a review over the last year or so (see here, or read the review's discussion document here). For those of you not in the know, the PBRF allocates some proportion of government university funding to each university on the basis of research performance. It is essentially a ranking exercise, undertaken by evaluating individual researchers (which is quite different from the UK or Australia, where the unit of assessment is the department), and then aggregating up to disciplines and to each university as a whole. It provides legitimacy to the claim that Waikato is number 1 in economics.

With the review underway, this article by Bob Buckle, John Creedy, and Ashley Ball (all Victoria University of Wellington), published in the journal Australian Economic Review (ungated earlier version here), is particularly timely. They looked at how the three previous full PBRF rounds (in 2003, 2012, and 2018, and ignoring the partial assessment round that occurred in 2006) affected New Zealand universities. Specifically, they look at the incentive effects, noting that:

A university can improve its research quality in three ways, although strong constraints are placed on the changes that can be made. Changes in average quality depend on the exits and entries of individuals (to and from other universities in New Zealand, or international movements), and the extent to which remaining individuals can improve their measured quality. A university can also influence its average quality by changing its discipline composition in favour of higher-quality groups.

They use confidentialised data from the Tertiary Education Commission on every researcher included in the three PBRF rounds to date, and focus on changes between 2003 and 2012, and between 2012 and 2018. Overall, they find some positive impacts on research quality:

There was a rise in the proportion of As and Bs in both periods... the proportion of A‐quality researchers increased by a factor of 2.5 between 2003 and 2018: from 6.5 per cent in 2003 to 13.3 per cent in 2012 to 16.4 per cent in 2018. The proportion of Bs increased by a factor of 1.5: from 25.9 per cent in 2003 to 39.6 per cent in 2012 and to 41 per cent in 2018.

A-quality researchers are world class in their fields, so an increase in researchers in that category is clearly a good thing. However, it does matter how the universities got there, and on this point Buckle et al. find that:

The net impact of exits on the AQSs [Average Quality Scores] is positive for all universities and disciplines in both periods; the net impact of entrants is always negative; and the net impact of QTs [Quality Transformations] is always positive.

In other words, the average quality of academics who exited the industry was lower than the average of those who stayed, while the average quality of new entrants (often early career researchers) was lower than the average of those already in the universities, and there was an improvement in measured quality among those who stayed. Looking at whether changes in individual researcher quality within disciplines, or changes in the composition of disciplines, contributed more to the improvement in measured quality overall, Buckle et al. find that:

For all universities combined, for 2003–12, the decomposition method found that all the increase in the AQS came from researcher quality changes. During the second period, the contribution arising from quality improvement was by far the dominant influence: indeed, the overall AQS would have been 3 per cent higher in the absence of any change in the overall discipline composition.

In other words, more of the improvement in AQS came from improvements in individual researcher quality scores, and not from universities making opportunistic changes to their disciplinary structures. 

So, overall, PBRF has had some good effects, and avoided the worst potential incentive effects for universities (at least at the disciplinary level). However, there are limits to how far those changes can be pushed. As Buckle et al. note in their conclusion:

The substantial reduction in the rate of quality improvement during the period 2012–18, compared with the earlier period 2003–12, suggests some streamlining of the process may be warranted, particularly in view of the high compliance and administrative costs. Furthermore, the major contribution to the average quality improvement in all universities and disciplines has resulted from the large number of exits of lower‐quality researchers. This also suggests that the extensive process, and information required, used to distinguish among higher‐quality categories is no longer necessary.

The PBRF is immensely time-consuming and costly, both to individual researchers who need to spend a lot of time preparing and polishing individual portfolios, and to universities. Researcher quality has improved tremendously in New Zealand since the PBRF was introduced in the early 2000s. The low-hanging fruit has clearly been picked, and the non-performing researchers are mostly gone. It is time to re-consider whether the PBRF remains useful, especially in terms of an assessment of individual researcher quality. Buckle et al. stopped short of conducting a full cost-benefit analysis of the PBRF system, but it is time that someone followed through and completed that exercise.

Iceland and the four-day workweek

Iceland has been in the news over the last couple of weeks for the success of a trial of a four-day workweek (see here and here, for example). As the BBC reported:

Trials of a four-day week in Iceland were an "overwhelming success" and led to many workers moving to shorter hours, researchers have said.

The trials, in which workers were paid the same amount for shorter hours, took place between 2015 and 2019.

Productivity remained the same or improved in the majority of workplaces, researchers said.

I took these reports at face value, while noting that my concerns about the Perpetual Guardian trial in New Zealand, especially in relation to the Hawthorne effect, remain valid. However, it appears that there has been some misreporting of what was actually trialled in Iceland. Anthony Veal (University of Technology Sydney) wrote in The Conversation today:

It almost seems too good to be true: a major trial in Iceland shows that cutting the standard five-day week to four days for the same pay needn’t cost employers a cent (or, to be accurate, a krona).

Unfortunately it is too good to be true.

While even highly reputable media outlets such as the BBC have reported on the “overwhelming success” of large-scale trials of a four-day week in Iceland from 2015 to 2019, that’s not actually the case.

The truth is less spectacular — interesting and important enough in its own right, but not quite living up to the media spin, including that these trials have led to the widespread adoption of a four-day work week in Iceland...

The media reports are based on a report co-published by Iceland’s Alda (Association for Democracy and Sustainability) and Britain’s Autonomy think tank about two trials involving Reykjavík City Council and the Icelandic government. The trials covered 66 workplaces and about 2,500 workers.

They did not involve a four-day work week. This is indicated by the report’s title – Going Public: Iceland’s journey to a shorter working week...

Read on to the third paragraph and you’ll learn the study “involved two large-scale trials of shorter working hours — in which workers moved from a 40-hour to a 35- or 36-hour week, without reduced pay”.

A four-day week trial would have involved reducing the working week by seven to eight hours. Instead the maximum reduction in these trials was just four hours. In 61 of the 66 workplaces it was one to three hours.

Extrapolating from the effect of a reduction of 1-3 hours (for the majority of employers in the trial) to a reduction of 8 hours may be a bit of a stretch. Veal also notes the potential for the Hawthorne effect - workers know that they are involved in a trial, and that researchers (and their employers) are watching them closely. It is only natural that they would work a little harder, and this would manifest in higher productivity.

It may be too early for this (from the BBC article):

The trials led unions to renegotiate working patterns, and now 86% of Iceland's workforce have either moved to shorter hours for the same pay, or will gain the right to, the researchers said.

At least though, we can watch with interest how a larger scale shift to shorter working hours affects productivity in Iceland. However, to understand that question we would need to agree on what we mean by productivity.

In general, labour productivity is the economic output per unit of labour input. In this case, it really matters how you measure labour input. If you measure it in terms of hours of labour, reducing the workweek for all workers might increase productivity, while at the same time reducing the total output of the economy. That arises simply because of diminishing marginal returns to labour. Each worker is progressively less productive each hour that they work than they were the hour before (maybe tiredness or boredom are factors here). So, removing the least productive hours from the workweek will raise average productivity, but would still mean that less work gets done in total.

However, if you measure labour input in terms of the number of workers (or equivalent full-time workers, even adjusting for the change in definition of 'full-time'), then labour productivity will decrease. Unless you genuinely believe that there is negative marginal product from the hours that are being cut, in which case the employers are irrational and should have cut hours long ago, without any need for government intervention (why would an employer pay a worker for hours that decrease their total output?).

There are other salient issues that I don't think have been adequately canvassed on this topic. For instance, in jobs where there are significant tournament effects, there might be no decrease in productivity measured per worker. That's because when there are tournament effects, people are paid a 'prize' for their relative performance (that is, for winning the 'tournament'). The prize may take the form of a bonus, a raise, or a promotion. The point is that each worker only needs to be a little bit better than the second best worker in order to 'win' the tournament. Those incentives would work to undo the decrease in work hours, since if everyone else reduces their work hours from 40 to 32, a worker that keeps working 40 hours will increase their chances of winning the tournament. If you doubt that tournament effects are real, I recommend asking any serious academic how many hours they work each week (since tournament effects are rampant in academia). This is not consistent with the overall goal of the four-day workweek, which is to reduce work. All it would do in these occupations is shift more of the work to outside of the paid workweek.

Also, the four-day workweek may be great for employees, but how will it affect self-employed workers? Or 'contractors', who are nominally self-employed but have little control over their work conditions. Or workers in the gig economy? Or interactive service workers (e.g. baristas), where the potential productivity gains (measured per hour worked) are likely to be close to zero? Will governments need to adjust the minimum wage (which is expressed in hourly terms, not weekly terms)? Does this reduce holiday entitlements (which are generally expressed as a number of weeks, but each week is now four days, not five)? That isn't to say that any of these issues is fatal for a four-day workweek proposal, only that they are things that any government will need to think about before such a proposal goes ahead.

Read more:

• Perpetual Guardian, four-day workweeks and the Hawthorne effect

When do children care about inequality?

When my ECONS102 class covers inequality, one point I make is that inequality may be a problem if it is associated with spatial segregation - that is, richer people living in richer areas, and poorer people liver in poorer areas. That is a problem because it may lead to fewer interactions between class groups, leading to lower sympathy for lower-class groups among upper-class groups, making those with power more accepting of increased inequality. That argument is based on research cited in the Max Rashbrooke book Inequality: A New Zealand Crisis. However, that research was (as far as I am aware) all based on adults.

In contrast, this new article by Kelly Kirkland (University of Melbourne), Jolanda Jetten (University of Queensland), Matti Wilks (Yale University), and Mark Nielsen (University of Queensland), published in the Journal of Experimental Child Psychology (sorry, I don't see an ungated version online, but there is a non-technical summary available on The Conversation), uses a cool experiment to test children's (aged 4-9 years) reactions to inequality. In the experiment:

Children played a series of games with puppets, where each accrued points over time, resulting in a context characterized by high or low inequality.

Children were asked whether they wished to donate some of their 14 sticker rewards to an unknown poor child. The current study revealed that older children donated more stickers compared with younger children. This finding is consistent with prior research suggesting that children generally become more altruistic with age... In addition, and contrary to the first hypothesis, children donated similar amounts of stickers regardless of which inequality condition they were in.

On the latter finding, children gave the same amount of stickers regardless of whether the game resulted in more, or less, inequality between the players. However, children's neighbourhood environment also mattered:

Notably, the inequality in children’s home suburb was linked to their donation behavior; children who lived in more unequal areas donated fewer stickers. This provides evidence that persistent real-world inequality may influence children’s prosocial behavior.

This finding accords with the research cited in Rashbrooke's book, but clearly needs replication beyond a single study of 120 children in Australia.

After the altruism task, the children also engaged in:

...a resource division task where they were given six extra points to divide among the puppets. Consistent with the second prediction, older children tended to give more to the poorer puppets compared with younger children. This aligns with previous work suggesting an increase in equitable resource division as children age... In addition, children were more likely to give the resources to poorer puppets after experiencing high inequality than after experiencing low inequality, and this effect was contrary to predictions.

It's not clear to me why that finding would be contrary to what you expected. If children have aversion to inequality, then you would expect a greater aversion to higher inequality. An interesting point was this:

...children’s justification for their resource division behavior revealed marked age shifts in their reasoning. Older children justified their behavior substantially more by referring to the ways in which the points were divided (e.g., ‘‘They were the ones that got the least amount of points”). On the other hand, younger children were much more likely to divide resources based on factors other than the point division (e.g., ‘‘I like that animal,” ‘‘That animal is cute”). Indeed, the effect of older children being more likely to give to the poor puppets was fully mediated by their tendency to refer to the point division.

So, inequality becomes more salient as children get older, and they also perceive it as "less ok". Anyway, the neighbourhood-level effects were most interesting to me. It would be interesting to see how those interact with household socio-economic status (which was included in the analysis, but provided not to be statistically significant on its own). For instance, is the effect of inequality on pro-social behaviour different between slightly richer children living in poorer areas and slightly poorer children living in richer areas? It is also interesting because it raises a question about how early in our lives our views on, and reactions to, inequality are formed, and how much they change over time and are shaped by our environment. More research like this is needed, but at this stage it seems to support the point that inequality may be a problem because of spatial segregation.

Read more:

• Who cares about income inequality?