The latest NZ research on social media and mental health is nothing special

If there are two lessons that I wish journalists could learn, they are:

1. That the latest published research is not necessarily the best research, and just because it is newer, it doesn't overturn higher quality older research; and
2. That correlation is not the same as causation.

This article from the Jamie Morton of the New Zealand Herald last week fails on both counts:

We blame friends' posts about weddings, babies and holidays for driving "digital depression" - but is social media really that bad for mental health?

A new study that dug deep into how platforms like Facebook, Twitter and Instagram influence our psychological wellbeing suggests not.

In fact, the weak link the Kiwi researchers found was comparable to that of playing computer games, watching TV or just minding kids.

The study is by Samantha Stronge (University of Auckland) and co-authors, and was published in the journal Cyberpsychology, Behavior, and Social Networking (sorry, I don't see an ungated version, but it appears to be open access). The authors used data from one wave of the New Zealand Attitudes and Values Survey, which is a large panel study that is representative of the New Zealand population. Using a sample of over 18,000 survey participants, they found that:

After adjusting for the effects of relevant demographic variables, hours of social media use correlated positively with psychological distress... every extra hour spent using social media in a given week was associated with an extra .005 units on the psychological distress scale. Notably, social media use was the second strongest predictor of psychological distress out of the other habitual activities, at approximately half the effect size of sleeping...

The coefficient is tiny. However, there are a couple of problems with this study. The first is pretty non-technical - this study is pure correlation. It tells you nothing about causation at all. That might not be a problem if the effect is zero.

However, the second problem with this study is that the authors simply dump a whole bunch of variables into their regression, without considering that many of these variables are correlated with each other. That leads to a high risk of multicollinearity, and the consequence of multicollinearity is that the coefficients are biased towards zero. In other words, you are more likely to observe a tiny effect simply because of the way they have analysed their data.

This research paper basically adds nothing to our understanding of whether social media is good or bad or otherwise for mental health. There are much higher quality studies available already (such as this one and this one).

This seems to me to be a real problem with the NZAVS research. They have a very large panel of data, and a huge number of measures covering lots of different domains. However, their approach to using this data appears to be 'throw everything at the wall and see what sticks'. However, that approach does not lead to high quality research, and even if it does lead to a large number of publications, they are mostly of dubious value, like this one.

Journalists could do with a bit more understanding on what constitutes a genuine contribution to the research literature.

Read more:

• What happens when you disconnect from Facebook?
• Does the Internet make people happier?
• The more you use Facebook, the worse you feel
• Quitting Facebook might make you happier