Statistics, lies and videotape
Mark Twain famously quoted 19th-century British Prime Minister Benjamin Disraeli as saying, “There are three kinds of lies: lies, damned lies and statistics”. (Apparently Disraeli didn’t actually say this, but it is still a good quote!)
It is a common and long held opinion that statistics are used to manipulate data to bolster weak and untrue arguments. So, does this mean you can’t believe any statistics that are quoted? How useful are they anyway? And what’s videotape got to do with any of this?
Statistical methods are used across the fields of business, science, medicine, engineering, sports, economics and politics (to name a few!), to help people identify, study, gain an understanding of and solve complex problems.
Whether through an observational study (such as a survey) or experimental investigation, the four basic steps, as highlighted in this useful online statistics textbook, are:
- Defining the problem
- Collecting the data
- Analysing the data
- Reporting the results
Often the results reveal more questions that need further study, but ultimately the results of a statistical analysis enable better-informed decisions to be made.
Lies told using statistics are often not intentional; there are many common errors, which can occur at any of the four basic steps defined above. The book How to lie with statistics, and many articles that can be found on the Internet, highlight the typical intentional and unintentional ways statistical methods are applied incorrectly, leading to wrongly drawn and misleading conclusions.
Some common errors are:
- Not defining which of the mean, median or mode is being used as the “average” value (Explained very well on How to lie and cheat with statistics.)
- Graphically presenting the data in misleading ways. (Some good examples on How to lie and cheat with statistics.)
- Not using a representative sample of the population, and then trying to make the results apply to the whole population. (A detailed American Statistical Society Pamphlet on how to conduct a well-considered survey discusses this and much more.)
- Consciously or unconsciously influencing the results in proving a particular point in line with the prejudices or preferences of the investigator. (A blog post which reviews a UCLA paper on conflicts of interest in pharmaceutical research.)
- Drawing wrong causal inferences. (Filip Spagnoli’s amusing blog post about this.)
- Simply making technical errors. (Such as those made recently when tests showed that neutrinos could travel faster than the speed of light.)
So we’ve had the statistics, the lies, and here is the “videotape”.
In this TED lecture (at 13:45min) Oxford mathematician Peter Donnelly describes how, at a trial in Britain, an expert witness (whose expertise was paediatrics and not statistics) incorrectly interpreted statistics in his testimony. The consequence of this was that a woman who had the misfortune of losing two babies to Sudden Infant Death Syndrome (SIDS) was convicted and sent to prison for murder.
Statistical analysis helps us to understand connections and patterns in complex data, which in turn allows informed decisions to be made. However, one must be careful when applying statistical methods to prevent nonsensical answers, and to ensure the result has not been manipulated to apply personal bias.
By Kim Harvey