Another installment in a nearly-regular series of mysteries and pseudo-mysteries (usually 3-4, but this time just one) this inquiring mind would like to have answers for:
QUESTION: Did this shake your faith in scientific reports and research like it shook mine?
It turns out that a major research paper relating to embryonic stem-cell research was faked:
Cloning pioneer to withdraw paper, doctor says
SEOUL (AP) â€” A doctor who provided human eggs for research by cloning pioneer Hwang Woo-suk said in a broadcast Thursday that the South Korean scientist agreed to withdraw a key research paper because most of the stem cells produced for the article were faked. (Related item: Independent probe sought).
Huh? I thought an “independent probe” is always done when major scientific findings are reported.
So let’s move to USAT’s “independent review” article linked above:
Science is at the center of this issue because important research papers are peer-reviewed and published in the prestigious journal.
….. In a letter released Tuesday by Science, eight prominent stem cell researchers â€” including Ian Wilmut, who cloned Dolly the sheep in 1997 â€” called for an independent verification of the Hwang lab’s cell-cloning feats.
Again, huh? Why the need for an “independent verification” if the paper was already “peer-reviewed”?
Mystery solved (HT Amy Ridenour)–”peer review” gives none of the assurance a layman would expect from the term (the excerpt concerns climate science, but blows open the fallacy in the general public’s understanding of all scientific “peer review”):
In business, â€œfull, true and plain disclosureâ€ is a control on stock promoters. While it may not always be successful, it gives an enforcement mechanism. There is no such standard in climate science. (or “science” in general–Ed.) ….. In fairness, the journals do not require authors to warrant full, true and plain disclosure and there is little guidance to such authors as to what is required reporting and what is not required.
Iâ€™ve found that scientists strongly resent any attempt to verify their results. One of the typical reactions is: donâ€™t check our studies, do your own study. I donâ€™t think that businesses like being checked either, but one of the preconditions of being allowed to operate is that they are checked. Many of the most highly paid professionals in our society â€“ securities lawyers, auditors â€“ earn much of their income simply by verifying other peopleâ€™s results. Businesses developed checks and balances because other peoplesâ€™ money was involved, not because businessmen are more virtuous than academics.
Then there is this comment at the same post:
I donâ€™t think that itâ€™s practical for journal peer reviewers to check every piece of data and every calculation – nobody would ever do reviews. But people should realize that even â€œpeer reviewedâ€ journal articles are (in business terms) unaudited. It doesnâ€™t mean that they are wrong; it only means that they are unaudited. The crunch comes if people rely on them as though they were audited.
So the next time you hear the term “peer-reviewed,” I would substitute these words: “passed the smell test (maybe, and if the person submitting the work is ethical and conducted his/her work conscientiously and honorably).”
Given the ever-larger dollars, very often tax dollars, that are based on the reliability of scientific work, standards must be raised, even if it costs money up-front (auditors, if you will) to raise them, and even if scientists’ egos are bruised in the process.
Maybe we as citizens should demand that the entire scientific community be brought under the heavy hand of Sarbanes Oxley.
Full Disclosure: I have recently posted on what I see as disparate media treatment between embryonic stem cell research and adult stem cell research (with the press reporting more favorably, and more often, on embryonic developments), and my personal belief that long-run success in using stem cells will come from the adult variety.
UPDATE: BizzyBlog links, you decide–”Cloning Doctor Finally Answers Critics — South Korean stem cell pioneer insists his groundbreaking research is still credible”
UPDATE 2, Dec. 26: What is probably the last word — “Stem cells in disgraced scientist’s paper did not exist: SKorean report”