Good news and bad news on a BBC report involving a scientific study

By Brian Clegg

In the past I have often criticised the media for referring to scientific studies without referencing them, and for treating weak studies as if they were some kind of scientific ‘proof’. There’s good news and bad news on this from a recent BBC report which avoided one problem, but fell into the other trap.

The report was about the reasons we apparently buy more from self-service screens (for example at McDonalds) than when dealing directly with a human being. In the text, North of England reporter Rowan Bridge spends quite a while describing what behavioural science is. The reporter writes:

Behavioural science can also impact your choice on more sinful things too. For example, your choice of wine. It may sound like an urban myth, but there’s real science behind it.

In one experiment, Adrian North and his colleagues at Leicester University alternated the music in a supermarket between French accordion and German oompah tunes.

On the days French music was played, French wine sales dominated, on German music days the German wine sold far better. But the effect was subconscious. When asked, customers weren’t even aware of the effect, but the numbers were clear.

The good news is that Rowan has included a link to this study as well as several other studies mentioned in the article. A gold star for this. The BBC hardly ever links to studies.

The bad news is that there is no questioning in the article of the validity of the ‘behavioural science’ presented. (I am just about to review a book called It’s On You, which is to some extent an exposé of poor behavioural science and the way it has been used.) I thought I would take a look at the wine sales paper and it has a number of red flags.

Firstly, it dates from 1999, well before the replication crisis meant that the majority of social science papers would prove doubtful at best. Secondly it’s a pitifully small sample size. Rowan tells us ‘the numbers were clear’ – they were certainly clearly small. Only 82 bottles were sold in the study (oddly, bought by 82 shoppers – it seems strange no one bought more than one bottle). Of these only 44 took part in the survey that determined if they thought they were influenced. The results did have p values better than 0.001, so were technically significant, but with such a small sample it’s hard to say anything was established other than the need for further research. 

Kudos to the researchers, incidentally, for pointing out that ‘Because the music was played from the display containing the wines, it may well have attracted the attention of people who already intended to buy wine from one of the countries in question, rather than as a direct influence on customers product selections.’ An observation not mentioned in the news article.

I don’t think the media should ignore studies like this, but they do need to apply some provisos, rather than take them at face value. It’s possible if the article had been written by a science correspondent rather than a North of England correspondent they would have done this. But sadly, even science writers can fall for this error.

Image from Unsplash+ by Getty Images – note this does not illustrate the actual experimental setup

These articles will always be free – but if you’d like to support my online work, consider buying a virtual coffee or taking out a membership: 

See all Brian's online articles or subscribe to a weekly email free here

Now Appearing is the blog of science writer Brian Clegg (www.brianclegg.net), author of Inflight Science, Before the Big Bang and The God Effect.