How not to write Popular Science

There was one issue that was specific to self-publishing: make sure there are no layout oddities. Appearance is as important as fixing typos. This particular book had a first paragraph in a smaller font that the rest of the text (as well as a couple of spurious bursts of italics, finishing part way through a word). More generally applicable, the author was a medical doctor and used ‘Doctor X Y’ as their author name. I strongly recommend not doing this – many popular science writers have doctorates, but very few use the title in their author name – it just looks tacky. (It’s even worse if, say, you are writing about physics but are a medic.)

The book covered a phenomenon that could have a physical cause, could be purely psychological or simply the result of lying (or any combination of the above) – important to know, as we will see, when looking at a survey that was central to the story. A first content lesson is that it’s really important to present numbers in an easily-followed way – if there is some oddity in the statistics, for example, then it needs to be carefully explained. In the survey, 50% of the studied population had a particular experience, but 75% of the population had a subset of the experience. This clearly can’t be literally true – it turned out to be because the survey was not particularly well worded, and a fair number of participants didn’t realise that the second type of experience was just one example of the first.

It’s also important only to state what can be logically deduced from a study. In this case, the author assumed because 200 experiences were described in the survey, there must be a physical explanation. Unfortunately, as suggested above, this is a false deduction. The fact that 200 people described subjective experiences does not mean that there anything physical happened – far more evidence than a self-reporting survey is needed to be able to make this deduction. It does mean there is something to investigate – but does not give any direction on what that might be.

We next hit on problems with probability. This is never an easy subject to deal with – if it’s outside your personal area of expertise, then it’s always worth getting expert guidance. The author works out an incredibly small likelihood of something happening by chance. Unfortunately, in doing so, she hits two familiar probability pitfalls. The first is of dismissing an occurrence as too unlikely to be coincidental when it happens to a specific person, where the relevant probability was the likeliness of it happening to anyone. It’s a bit like saying the probability of me winning the lottery at 16 million to one is almost zero – but the reality is that someone wins most weeks. Similarly, I’ve more than once bumped into someone I know in a different country from the one where we both live. The probability of both that specific person and me being in that place at that specific time is ridiculously small. But the chances of such a chance meeting happening at some time is large.

The second probability error is the probability equivalent of the false deduction from the survey: the false dichotomy. The author mentions p values – the probability of something happening if the null hypothesis applies. But it’s essential to remember that this is not the probability that your hypothesis isn’t true. If it’s very unlikely that there is no cause for something occuring, it doesn’t mean that your theory for what that cause is happens to be true. Again, I’m not saying there isn’t something worth investigating further, but there is no evidence by suggesting the null hypothesis is unlikely that your preferred reason is correct. 

Later on we get some physical science, but unfortunately the author has limited knowledge of physics and gets enough wrong to make the whole proposition open to doubt. We are told that humans ‘generate 25 watts per second in our brain alone’. Unfortunately a watt is a measure of energy per second already, so ‘watts per second’ is meaningless unless we’re talking about a rate of change. Also, the brain doesn’t generate 25 watts of electrical power, it consumes about 20 watts, not all of which will be translated into electrical energy. We are also told that neurons have a ‘negative electrical charge at rest of minus 70 millivolts/millimetre.’ Leaving aside voltage not being a measure of charge, we are told this is equivalent to 14 million volts per metre, which makes it more powerful than lightning. Even if the numbers were correct, this is highly misleading, both because the neuron’s potential of 70mV isn’t measured across a millimetre, and because you can’t multiple the potential difference up meaningfully in the way it was done here. 

A final major issue is the haphazard use of studies to back up a theory. Bearing in mind we are dealing with a topic that could both involve physical and psychological effects, there is no mention of the replication crisis in psychology and plenty of use of psychology studies going as far back as the 1970s. Some of these have been thoroughly discredited (particularly where physicists were trying to work outside their own field) and others have repeatedly failed to reproduce. There was a time when even high profile popular science titles (for example Daniel Kahneman’s Thinking Fast and Slow) could get away with using studies from the swamps of pre-2012 psychology that would later prove unreproducible, but that should no longer be the case.

Some insist that an author should be an expert in the field they are writing a popular science book about. I disagree (I would have to, given the range of books I’ve written) – but if you are stepping outside your areas of expertise, you do need to be extremely careful about these kind of issues if you are to produce a title that does justice to the topic and doesn’t mislead the reader.

See all Brian's online articles or subscribe to a weekly email free here