Tom Olijhoek notes that
Citation rates are still commonly used for the assessment of the quality of individual scientists and for the assessment of the quality of individual scientific journals. In the latter case the measuring tool, the impact factor (IF) is thought to represent the chance for high citation rates when publishing your work in High Impact journals. High citation rates for articles are thus taken to mean high quality for the underlying science. In reality the impact factor has been shown to correlate poorly with actual citation rates
He argues that open access opens ways for “far” better methods for the assessment of scientific quality.
One of his points counts against the climate denial squad – that the more spectacular or unusual the results are, the more the chance is that they will be accepted for publication in leading scientific journals. Thus the journals are inclined to undermine consensus, not to defend it.
I’m sympathetic to the idea that review should happen mostly after publication, allowing for more participation by outsiders to a given field. For me, the price paid by the climate science establishment will be worth it on the day that a similar price is paid by the economic theory establishment. But how the review process would work in the absence of a consensus is far from obvious. The main thing to remember is that there is a baby in all that bathwater.