Limits of peer review
We already know peer review isn’t a panacea. For one, it is widely known that peer review simply cannot prevent scientific fraud, because as experimentalists, we tend to trust in the data—we may question the process of data gathering, but once the process itself seems free of error, we trust the data as presented, unless it claims something fantastically impossible, usually involving violation of energy conservation or something on that order.
But, recent global warming scandal, a.k.a. ClimateGate, has uncovered even more problems with peer review: it’s done by “peers”
In response to an article challenging global warming that was published in the journal Climate Research, CRU head Phil Jones complains that the journal needs to “rid themselves of this troublesome editor”-hopefully not through the same means used by Henry II’s knights. Michael Mann replies:
I think we have to stop considering “Climate Research” as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal.
Note the circular logic employed here. Skepticism about global warming is wrong because it is not supported by scientific articles in “legitimate peer-reviewed journals.” But if a journal actually publishes such an article, then it is by definition not “legitimate.”
In short, if your peers are crackpots, would you want your work reviewed (and judged) by those peers? Peer reviews are used to keep crackpots out of legitimate science journals—but, if the crackpots have already taken root inside that fence, then, well, crackpots can use it to keep legitimate science out of journals.
I wouldn’t call these failures of peer review necessary … failure of science. It represents pitfalls and setbacks of frontiers of research. Eventually (on the time scale of decades or even centuries), scientific theories are judged by their ability to produce predictions that no other theories can. And short of time travel, there is no way to fake that.