I came across two interesting papers recently which both dealt with why ‘bad’ science persists – it has been known for a long time that many studies are statistically underpowered, leading some people to famously state that ‘most research findings are false’. But why – when we as scientists know better – do we not do better? The answer lies in the incentives: ‘smart people do dumb things for smart reasons’!
The smart reason being that doing dumb things is incentivised and rewarded – in terms of publications and career progression.
One of the papers framed this from an evolutionary perspective, and the other from animal behaviour, trying to find the scientists’ optimal behaviour given the current circumstances. Both are worth a read! The two papers are
- The natural selection of bad science by Smaldino and McElreath
- Current incentives for scientists lead to underpowered studies with erroneous conclusions by Higginson and Munafo
The first one states that ‘… some of the most powerful incentives in contemporary science actively encourage, reward and propagate poor research methods and abuse of statistical procedures. We term this process the natural selection of bad science to indicate that it requires no conscious strategizing nor cheating on the part of researchers.’ The second one concludes that ‘Perversely, current incentive structures may promote low-quality science, because the research strategy they encourage is more likely to produce striking (but erroneous) findings’.
Both papers together make it very clear that more lamentations about the ‘bad’ science (such as underpowered studies, p-hacking etc.) are not going to change anything as long as the incentives stay the same. As long as doing bad science is rewarded, smart people will do bad science. Higginson and Munafo include some concrete suggestions of what could be changed to change the incentive structure: introducing more stringent peer review on statistical power, ‘considering more of a researcher’s output… and giving less weight to strikingly novel findings’. The problem needs to be attacked at the root – the incentives.
Higginson, A.D., and Munafò, M.R. (2016). Current Incentives for Scientists Lead to Underpowered Studies with Erroneous Conclusions. PLOS Biol. 14, e2000995. link
Ioannidis, J.P.A. (2005). Why Most Published Research Findings Are False. PLoS Med. 2, e124. link
Smaldino, P.E., and McElreath, R. (2016). The natural selection of bad science. R. Soc. Open Sci. 3, 160384. link