How not to measure the long term consequences of the Hiroshima atomic bomb

In a new article published in the Journal of the Japanese and International Economies (sorry, I don't see an ungated version online), Satoshi Shimizutani (Nakasone Yasuhiro Peace Institute) and Hiroyuki Yamada (Keio University) look at the long-term impact of the Hiroshima atomic bomb blast in 1945. They use data from the 2011-12 and subsequent waves of the Japanese Study on Aging and Retirement (JSTAR), along with a supplementary survey conducted in 2017. Their sample includes 653 people living in Hiroshima in the JSTAR sample, 297 of whom are defined as 'affected' by the Hiroshima bomb (primarily either because they were survivors of the Hiroshima bomb, or because they parents were). Comparing those two groups across a wide range of socio-economic variables, they find that:

...more than 60 years after the tragic event, survivors and their children are not seriously disadvantaged in marriage status or educational attainment but some significant distinctions between the affected and the non-affected group is observed in such aspects as combination of married couples, work status, mental health, and expectations.

Specifically, they find that affected people are more likely to have inherited their house, affected women (but not men) are more likely to be self-employed, work in small firms, and hold managerial positions, while affected men (but not women) report a lower subjective probability of living to age 85, and more depressive symptoms. There are a whole range of variables where the differences are not statistically significant.

There are a couple of problems with this study. The first is that they studied a wide range of socio-economic variables, but made no adjustment for the multiple comparisons that they made. Simply by chance, five percent of all comparisons are likely to be 'statistically significantly different' at the five percent level of significance. The fact that they don't really observe any effects that are consistent for both men and women should also be a big red flag here. Why would there be employment effects for affected women, but not men? Shimizutani and Yamada don't provide a strong theoretical reason to support their results, and fail to acknowledge the big limitation on them that the multiple comparisons creates.

The second, and probably more serious, problem is survivorship bias. The majority of the worst-affected people won't have survived to 2011, in order for their data to be included. Only the long-term survivors are included in the sample. So, at best this study can say that it looks at the long-term consequences of the Hiroshima atomic bomb, for those that survived to at least 2011. The long-term effects on the whole population affected in 1945 or thereafter were likely much greater, and much of those effects are unobserved. Again, this limitation wasn't acknowledged by Shimizutani and Yamada.

The short-term and long-term health consequences of the atomic bombings in 1945 have been extensively studied. The socio-economic consequences have received much less research attention. Unfortunately, this study doesn't do an adequate job of filling that gap.