LATEST ALERTS
Whatever Mark Twain or Benjamin Disraeli may or may not have said on the subject, statistics are not in themselves a species of lie. In fact, they are at the heart of pretty much everything we do.
But while statistics are vital to understanding our world, they are often misunderstood. Which may not be because they’re being manipulated by liars – though that is sometimes the case – but because they are both open to interpretation and, let’s face it, sometimes difficult to interpret. Whether deliberately or not, the same statistical findings can often be presented to show very different pictures indeed.
And that can be hugely important when it comes to forming opinion in the minds of both the general public and the legislators who decide what should be allowed, what should be regulated, and what should be banned.
Take these figures published recently by Statistics Netherlands, the official Dutch statistical office, showing a big jump in cannabis use by children aged 12-16. If they are to be strictly believed, more than twice as many Dutch kids in that age bracket used cannabis in 2022 as in 2021 – an increase of 115%. If you don’t like the idea of people that young getting high, that could sound scary.
Then again, 12-16 is a somewhat arbitary grouping – how many of those new cannabis users have just turned 12, and how many are going on 17?
And how different does it sound if we report it as a mere 1.5 percentage-point increase, from 1.3% to a still fairly moderate 2.8%? It’s an equally valid way of reporting exactly the same figures. And, in any case, without knowing how large the sample was in each year, it’s impossible to guess how representative the results really are of the young Dutch population overall.
US debate and controversy
So how about these figures from the National Institute on Drug Abuse, an official source of science statistics in the US? Taken from the national Monitoring the Future study, they show what the report calls a “significant” increase in cannabis use by young people.
You could say past-year youth use increased nearly 50% over ten years, while the number of those using it daily increased by more than 80%, which sounds pretty high. You could say almost half as many students again tried it in 2021 as in 2011 and the number using it daily nearly doubled, which possibly sounds even more startling. Or you could say past year use increased by 14 percentage points, from 29% in 2011 to 43% in 2022 while daily use increased by six percentage points from 5% to 11% over the same period.
All equally valid ways of reporting exactly the same figures. And the picture does look fairly remarkable however you look at it. It has led to a debate over whether youth use of cannabis is increasing and whether it is increasing faster in states that have legalised cannabis products.
Smart Approaches to Marijuana (SAM), a group opposing the legalisation and commercialisation of cannabis, has said legalisation is leading to higher rates of youth use. The group recently discussed a study published last month that used data from different, older US national drug use data sets (the available statistics on cannabinoids are often out of date by the time they are scientifically analysed).
“Cannabis use was on a decades-long decline thanks to the concerted work of prevention efforts, but the legalisation and commercialisation of cannabis is threatening to erase those public health gains,” according to SAM president and CEO Kevin Sabet. “We also can’t ignore the fact that young people are beginning to use new forms of today’s super high potency cannabis, such as vapes and concentrates, that are significantly linked to serious harms to mental health and higher rates of addiction.”
If you’re looking for bad news…
In reality it is not clear whether increases in youth use are really linked to legalisation at all, and there is no definitive consensus finding through surveys that it is increasing faster in states where cannabis is legal. After all, a legal market naturally has better purchaser age verification techniques than an illicit market. And young people are probably more likely to report use – even when data collection is anonymised – when it is a legal product in the state, even if not for their age group.
Nonetheless a simple headline reading of youth increases would potentially feed into a narrative such as that presented by SAM.
Let’s look at how some other contentious figures are reported. As our sister service ECigIntelligence reports this week, a study newly published in the Journal of the American Medical Association (JAMA) found that young people who use e-cigarettes are more likely to become regular smokers than those who don’t. If you’re on the lookout for bad news, there’s your headline.
On the other hand, the numbers involved are nowhere near enough to alter the overall picture that smoking is simply going out of fashion with US youngsters. And indeed, of the very few vapers (and non-vapers) who do “try” combustible cigarettes, an almost vanishingly small number like smoking enough to go on doing it for long.
…but then on the bright side…
As Lion Shahab, co-director of the Tobacco and Alcohol Research Group at University College London (UCL), pointed out, the statistics can be reported in starkly different ways. And the temptation for many will be to use them to exaggerate enormously the actual significance (as opposed to the statistical significance) of what they reveal.
You could say they showed that kids who vaped first were 81% more likely to try smoking than their non-vaping classmates. Or you could examine the small numbers in addition to the big percentage, weigh up other relevant factors, and find that the absolute risk difference was a mere 0.88%. Not such an eye-catching headline.
In the words of the study’s authors: “Although the significantly higher odds of continued smoking among e-cigarette users suggest a potentially important problem, the small magnitude of absolute risks and the minor risk differences in continued smoking between baseline e-cigarette users and non-users indicate a much less consequential problem: few adolescents are likely to report continued smoking after initiation regardless of baseline e-cigarette use.”
Of course tobacco use and cannabis use are not the same thing – even though groups such as the Truth Initiative describe them as “closely linked”. But the main take away from the JAMA study could be applied equally to both areas: Statistical significance does not necessarily mean actual real world significance.
– Aidan Semmens and Freddie Dawson CannIntelligence staff
