This article is part of a series of essays I’ve written for an Introduction to the Philosophy of Science course at KTH on the fall of 2017.
Let’s imagine two new research papers coming out. The first one is a breakthrough paper talking about an exciting and new phenomenon. And the second one simply confirms the findings of a previous paper by reproducing it.
Then, let’s ask ourselves the following question: which one of them is most likely to be read first?
We can be as optimistic about researchers as possible, but if there’s one thing we unfortunately do know, it is that people get easily more drawn into learning new knowledge than merely confirming past one.
This simple fact may very well seem harmless at first sight. After all, isn’t science all about being at the edge of human knowledge and pushing the boundaries a little further? Isn’t science supposed to marvel the human brain?
Unfortunately, there’s one little overlooked fact: if science has allowed for great progress in the world, it was not only by producing new research papers, but also because it has always attempted to reproduce the experiments that allowed for the first paper’s claims to even be drawn.[1]
With this precious part of science gone missing, the scientific community has been recently facing a quiet crisis.
In an ideal setting, science corrects itself. So each paper coming out is an invitation for other fellow researchers to try to reproduce it, and debunk it if possible. But in practice, the incentives for scientists to actually put up the work into trying to reproduce each paper are not nicely aligned.[2]
Researchers are first and foremost human beings, who desire recognition for their work, and since new and exciting findings are always more advantageous for them to publish, we may risk ending up with an immense clutter of research findings that yield no significant progress because they are irreproducible.
If science is becoming more and more irreproducible, are certain disciplines more affected than other ones?
Research attempting to find new ways of treating cancer is proving to be a good candidate. Scientists at Bayer HealthCare, the medical branch of a multinational chemical company, have found that only six out of 53 published cancer findings were actually reproducible.
The researchers couldn’t replicate any of the findings, and even found evidence for there being simply no relationship between the studies’ conclusions and the data analysed.
An answer as to why this field seems to suffer from this problem more than other ones, would be the giant (and growing) biomedical research industry that encourages researchers to find new treatments but push them far less to confirm if old ones are working.
Even if researchers are less incentivized to reproduce old research, drug companies nevertheless possess huge amounts of data about which research bears out, and which one doesn’t. But unfortunately, these companies never disclose this data in fear of bad marketing.