Parachutes may be ineffective as safety devices

A lot of what you read about research in mainstream media is taken from press releases (by universities or research institutes), or at best, is taken from the abstracts of research papers. Randomised controlled trials are the gold standard in evaluation research (especially in health), so many readers might not take a critical eye when reading the studies. This would be a mistake.

A good example of where a lack of critical reading would go horribly wrong is this article from the Christmas issue of the British Medical Journal, by Robert Yeh (Harvard Medical School) and co-authors, entitled "Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial". Here's the first sentence of the conclusions from the abstract:

Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention.

One could conclude from that sentence that parachutes are ineffective as safety devices. However:

Compared with individuals screened but not enrolled, participants included in the study were on aircraft at significantly lower altitude (mean of 0.6 m for participants v mean of 9146 m for non-participants; P<0.001) and lower velocity (mean of 0 km/h v mean of 800 km/h; P<0.001).

The authors tried to enrol people into their randomised controlled trial by asking people:

...whether they would be willing to be randomized to jump from the aircraft at its current altitude and velocity.

They would be randomised into making the jump either with, or without, a parachute. The only people willing to be randomised in the study, unsurprisingly, were those on stationary aircraft on the ground. This limits the external validity of their sample a little, but does allow them to conclude that:

...although we can confidently recommend that individuals jumping from small stationary aircraft on the ground do not require parachutes, individual judgment should be exercised when applying these findings at higher altitudes.

This study was, of course, a response to an early BMJ article on parachutes from 2003, which concluded that:

...the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials.

Well, now the effectiveness of parachutes has been evaluated, and found wanting. Read both papers (they're open access); they're hilarious (as has been the case with many past papers in the Christmas issue of the British Medical Journal, such as this one I blogged about in 2017).

[HT: Thomas Lumley at StatsChat, whose pet hate is journalists quoting uncritically from press releases or abstracts of non-peer-reviewed papers]