Don’t believe everything you read: a cynical rule for cynical times. One might think the academic journal is the one place where accuracy, validity and integrity are still core publishing values. But the growing number of journals available on the market, as well as increasing pressure on academics to be published, has led to cracks in the system.
The challenges are twofold: aside from the quality of lecturing, tutoring and research skills, the modern academic’s reputation and career progression are dependent on how frequently they publish material. In the race to get into the best journals, the goal of producing high-quality, comprehensive research can be compromised by sloppy practices, which in turn can result in some experiments being irreproducible. In a small number of cases, even fraudulent material has been accepted.
It’s not just sloppy or dishonest research practices that are to blame. Academic journals have become the ultimate destination for research. What makes it into these bastions of credibility depends on the scrutiny of a select number of dedicated, anonymous academics working pro bono purely for the sake of knowledge.
Proliferation of journals
The number of academic journals has increased in recent years, while the number of “referees” – academics responsible for peer review – have not. So the market has never been more competitive. Similar to newspapers chasing the next big scoop, every journal wants to be the first to publish the newest “discovery” or “breakthrough”, particularly in the sciences. But this may lead to questionable research being published before it has been fully scrutinised.
"Even the big journals are susceptible to publishing erroneous research," says Prof Luke O'Neill of the Trinity school of biochemistry and immunology and editorial board member for the journal Science. "It is in their interests to publish the breakthroughs, and if on occasion a breakthrough is dramatic, big journals like Nature or Science will rush to publish it.
“From an academic’s perspective, in Ireland at the very least we still have a tenure system,” he says. “In the US your job depends on performance. If you don’t deliver you’re fired, and delivery is increasingly dependent on how published you are. So that creates an environment which gives rise to mistakes, sloppiness, even falsification.”
Academics, particularly in the sciences, enjoy a good reputation in society, one that perhaps assumes more honesty, integrity and discipline than is deserved.
“You must remember,” says O’Neill, “scientific research is a human activity, and humans are prone to certain behaviours: egoism, vanity, laziness, dishonesty. Scientists aren’t immune to it.”
Reproducibility
It has been suggested that up to 50 per cent of research in scientific laboratories can’t be reproduced. This is most prevalent in biology and the life sciences.
In some cases, it’s down to complex experimental processes being extremely difficult to recreate in another lab. In other cases, it’s down to errors and sloppiness. This is problematic, as one fudged figure can render new research useless to anyone else hoping to build upon it.
“Sloppy science is costing people money, time and effort,” says O’Neill.
"There's a lot that might have happened prior to the submission of a paper – certain errors, mistakes, misconduct – which can't be checked," says Dr Padraig Murphy, lecturer and programme chair of the MSc in science communication at DCU.
“It varies from subject to subject,” he adds. “In the physical sciences you might have huge amounts of experimental data, and there simply isn’t the time, resources or expertise to repeat procedures.”
Scientific Fraud
Instances of out-and-out falsification are rare, but it’s difficult to know how much is slipping through the cracks.
“In the last 10 years there’s been a number of outstanding international cases where people didn’t just get it wrong, they fraudulently invented results,” says Cormac O’ Raifeartaigh, lecturer in physics at Waterford Institute of Technology. “The big question is whether we take it that the researchers we’ve actually caught are the tip of the iceberg or that it is in fact a tiny minority. My opinion is that it is the latter. Science tends to self-correct quite well.”
Knowing just how much published material is either falsified or subsequently proven to be incorrect is difficult to measure. “The only way to look at this is by looking at the number of retractions being made, but publications are reluctant to give too much info about errors,” says Murphy.
Of course, journals will publish retractions if there is clear evidence against a questioned article. Understandably, they don't like doing it. There are others on the lookout, though, such as US-based blog Retraction Watch (Retractionwatch.com).
Any solutions?
The peer-review system assumes academics will anonymously scrutinise new research before it is published. An ever-increasing workload has made it more difficult, as it is done on a pro bono basis. So could referees be paid?
Murphy doesn’t believe that introducing payment to the referee system would help.
“There is a certain collegiality among academics in that they are doing their work for the sake of knowledge,” he says. “It’s part of their job to be reviewing others. There’s a certain noble aspect to the peer-review system. Introducing payment would change that.”
O’Neill disagrees. “I think paying the referees would be a good idea.That way you would appeal more to their consciences. Plus more would be willing to do it if they got paid.
“I also think anonymous referees are a bad idea,” he says. “If your name is on it you wouldn’t want to be accused of sloppy reviewing. In essence, we should make the reviewing process more transparent. That way it might be easier to spot the potential for sloppiness and fraud.”
BAD RESEARCH: HIGH-PROFILE INTERNATIONAL CASES
Riken Center for Development Biology In January, researchers at the Riken Centre for Development Biology in Kobe, Japan, published two studies describing how specialised cells could be changed into an embryonic-like state by stressing them. The research offered an unexpectedly simple way to make master stem cells for treating all manner of diseases.
However, in July, a retraction was published in the journal Nature, where it was revealed that an investigation by Riken found significant errors and “inexplicable discrepancies” that “impair the credibility of the study as a whole”.
Tragically, in August a Japanese scientist who co-authored the work, Yoshiki Sasai, hanged himself at the research centre where he worked.
Andrew Wakefield Many will already be familiar with the case of Wakefield, a former British surgeon and medical researcher. In 1998 he published a paper in the Lancet supporting a now-discredited claim of a link between MMR vaccinations and autism. It took the Lancet until 2004 to issue a retraction, and to this day the fraudulent research still influences some parents' decision about whether or not to vaccinate.
Hwang Woo-suk In 2004 and 2005, Korean veterinarian and researcher Hwang Woo-suk had papers published in Science based on a series of fabricated experiments in stem-cell research. He claimed to have succeeded in creating human embryonic stem cells by cloning. In 2006 he was convicted of embezzlement and bioethics law violations after it was found that most of his research was made up. Until then he was considered a pioneer in the field.
Jan Hendrik Schön German physicist Jan Hendrik Schön became notorious at the beginning of the 21st century when he claimed to have made a number of breakthroughs with semiconductors. Although his findings were published in various high-profile journals, no other research outfit in the world was able to successfully reproduce the results claimed by Schön. Before being exposed, he received a number of awards from his peers.
Jon Sudbø This Norwegian was exposed for scientific fraud in 2006. Over years he made up results in oncology that were published in a number of journals. The article that finally exposed him (published in the Lancet) was based on 900 test patients he had completely fabricated. The then editor of the journal described it as the biggest scientific deception ever carried out by one single researcher.
Word generators About 120 papers published in recent years were found to have been created by an automated word generator that put random, complex-sounding words in vaguely plausible sentence structures. The false papers were all maths- and computer science-related, with titles such as An Evaluation of E-Business with Fin and Simulating Flip-Flop Gates Using Peer-to-Peer Methodologies.
