The Science of Getting It Wrong (and how to make it right - eventually)

extracts below:

If the press release article seems important to you, it would be best if
you read the entire article in the link.

My takeaway is that if it is an important result, it should be
rigorously duplicated and reduplicated. Not accepted and falsely
circulated as official dogma.

================================================== =====


February 27, 2007
The Science of Getting It Wrong: How to Deal with False Research Findings

The key may be for researchers to work closer and check one another's
results

http://www.sciam.com/article.cfm?art...43F150B0DE1355

By JR Minkel

FALSE POSITIVES: Researchers poring over their samples for novel
results may be contributing to a flood of false research results.
Tighter collaboration between investigators may be one way to reduce
such errors.

Talk about making waves. Two years ago medical researcher John Ioannidis
of the University of Ioannina in Greece offered mathematical "proof"
that most published research results are wrong. Now, statisticians using
similar methods found—not surprisingly—that the more researchers
reproduce a finding, the better chance it has of being true.

Another research team says researchers have to draw conclusions from
imperfect information, but offers a way to draw the line between
justified and unjustified risks.

snip
In his widely read 2005 PLoS Medicine paper, Ioannidis, a clinical and
molecular epidemiologist, attempted to explain why medical researchers
must frequently repeal past claims. In the past few years alone,
researchers have had to backtrack on the health benefits of low-fat,
high-fiber diets and the value and safety of hormone replacement therapy
as well as the arthritis drug Vioxx, which was pulled from the market
after being found to cause heart attacks and strokes in high-risk patients.

Using simple statistics, without data about published research,
Ioannidis argued that the results of large, randomized clinical
trials—the gold standard of human research—were likely to be wrong 15
percent of the time and smaller, less rigorous studies are likely to
fare even worse.

[NB - such as many "quick and dirty "preliminary investigations"]

snip

"I fully agree that replication is key for improving credibility &
replication is more important than discovery," Ioannidis says.....

But Ioannidis left out one twist: The odds that a finding is correct
increase every time new research replicates the same result, according
to a study published in the current PLoS Medicine. Lead study author
Ramal Moonesinghe, a statistician at the Centers for Disease Control and
Prevention, says that for simplicity's sake his group ignored the
possibility that results can be replicated by repeating the same biases.
The presence of bias reduces but does not erase the value of
replication, he says.

Ioannidis says that researchers have become increasingly sophisticated
at acquiring large amounts of data from genomics and other studies, and
at spinning it in different ways—much like TV weathercasters proclaiming
every day a record-setting meteorological event of some sort. As a
result, he says, it is easy to come up with findings that are
"significant" in the statistical sense, yet not scientifically valid.

snip


Ioannidis agrees that perfect certainty is impossible. "If you have a
severe disease and there is only one medication available, and you know
that it is only 5 percent likely to work, why not use it?" he says. But
implementing such a calculus is trickier than it appears, he adds,
because "we cannot assume that an intervention is necessarily safe in
the absence of strong data testifying to this."





http://www.sciam.com/article.cfm?art...43F150B0DE1355