Call to ban alcohol sales in supermarkets is jumping the gun

Stuff reported last Wednesday:

New Zealand children are being exposed to alcohol nearly every time they go to the supermarket, sparking a call from researchers to have it banned from such stores. 

The over-exposure of alcohol to children put it on par with everyday products such as bread and milk, causing children to drink alcohol earlier in their life, Tim Chambers from Otago University's Department of Public Health said. 

The department's research found that 85 per cent of children were exposed to alcohol in Wellington supermarkets. 

The original research paper, by Chambers et al. and published in the journal Health & Place, is available here (sorry I don't see an ungated version anywhere).  The idea behind the research is kind of cool (albeit a bit Big Brother-ish) - the researchers set up children with GPS trackers that tracked location every five seconds and a wearable camera that took a picture every seven seconds, over a four-day period between July 2014 and June 2015. They then geo-located those images to supermarkets and coded the captured images in supermarkets as to whether or not they contained exposure to alcohol promotions. It's a pretty cool idea, but there are a number of problems with the analysis.

First, about 23% of the GPS location data was missing, so had to be imputed - that means that they replaced the missing values with a non-missing value, based on a Python script that used information "on spatial and temporal parameters". In spite of that, they still had to impute some of the missing data manually using "identifiable information in the images".

Second, the children were not very representative of the population, with 40% European, 36% Maori, and 25% Pacific (and no Asian children), and 44% from the lowest three school deciles. You can probably expect these children to have somewhat different experiences in terms of exposure to alcohol from Asian children and possibly from those in the middle four deciles (that made up just 19% of the sample).

Third, the sample size was small - only 168 children, of which only 56 children made at least one trip to a supermarket. So, to say that "85 per cent of children were exposed to alcohol in Wellington supermarkets" (the quote from the Stuff story) is just plain wrong. It was only 85 percent of 56 of the 168 children, or 29 percent of the full sample.

They then base a quantitative analysis on only a subset of this already small dataset, which unsurprisingly shows nothing statistically significant. The small sub-sample means that any statistical analysis is going to be underpowered.

However, the biggest problem with the study is their misunderstanding of the practical aspects of the legal changes to the way supermarkets sell alcohol, after the implementation of the Sale and Supply of Alcohol Act 2012 (SSAA). The Act came into force on 19 December 2013, and restricted supermarkets and grocery stores to selling alcohol in a single area. The appropriate section of the Act is s112(5):

The authority or committee must describe an alcohol area within the premises only if, in its opinion,—

(a) it is a single area; and

(b) the premises are (or will be) so configured and arranged that the area does not contain any part of (or all of)—

(i) any area of the premises through which the most direct pedestrian route between any entrance to the premises and the main body of the premises  passes; or

(ii) any area of the premises through which the most direct pedestrian route between the main body of the premises and any general point of sale passes.

The authors are correct when they note that supermarkets had a period of up to eighteen months in which to implement such single areas. However, that period didn't commence when the legislation became operative, but would commence on the date when each supermarket was granted a licence renewal. To extend this further, under the direction of the Alcohol Regulatory and Licensing Authority, all District Licensing Committees (the initial decision-makers on licensing issues) were asked to delay the hearing of any supermarket licence applications until after the appeal of the first such decision (known as the Vaudrey decision) was determined. That did not happen until earlier this year, and so the first supermarkets subject to single alcohol areas would not have been operating these areas until earlier this year (although some may have voluntarily imposed such conditions on themselves earlier, in practice few have done so).

So effectively, none of the supermarkets that the children in this study visited in 2014-15 were subject to the new laws in terms of single alcohol areas. That means that the analysis in the research paper, where the authors compared exposure to alcohol in one supermarket, before and after a change in the supermarket's configuration, tells us absolutely nothing about whether the new law has had any effect on exposure of children to alcohol in supermarkets. That analysis was based on only sixteen visits (11 in 2014, and just five visits in 2015). Which means it is a huge over-step for the authors to conclude that:

In a case study within this research, the current 2012 SSAA was ineffective at preventing children's exposure to alcohol marketing at supermarkets.

You simply can't tell whether that is the case based on this research. They could re-run their research now that many supermarkets are actually implementing these areas, and see if things are better (but in that case they should use a bigger sample size than a few children if they are really genuine about quantitatively testing whether there is any impact). However, the authors clearly had their conclusions in mind before they began the study, because they also conclude that:

Banning alcohol sales in supermarkets appears to be the only way to prevent such exposure.

If your goal is to prevent any exposure of children to alcohol, then why bother going through the expense of doing this research? You could simply say: (1) Any exposure to alcohol is bad; (2) there is alcohol in supermarkets; therefore (3) to prevent any exposure of children to alcohol, you must ban alcohol sales from supermarkets. You wouldn't even need to dress it up as a research paper, since it is so obvious.

[HT: Marcus]