If the editorial and referee processes were perfect, there would be very little role for the CFR. The best papers would go into the best journal; the second-best into the second-best journal, etc. However, I believe there is evidence that the referee process is noisy and decisions on papers are partly subjective. There are many good papers that fall through the referee process (and a few bad ones that slip through!).

I have recently written a paper on Referee Recommendations, published in the RFS. Referees provide more of an idiosyncratic signal than a common signal. Compared to existing journals, the CFR trades away more referee idiosyncracy for more editorial opinions—but it is a matter of degree, not black-and-white. The CFR does consult referees and other journals have editorial opinions, too.

As to more readings about refereeing, let me suggest Lynch Reviewing, and J. Scott Armstrong, Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation. Armstrong worries that refereeing noise means that it is not worthwhile writing controversial papers. He suggests inviting more papers, providing some early acceptances, allowing authors to nominate referees, favoring non-result-finding papers, using structured rating sheets, open peer review, and electronic publication. I do not follow most of these recommendations, but I agree that the current referee process tends to disadvantage important, controversial, and interesting papers, and that it is the job of a good editor to use the help of referees appropriately.

/2015/01/01/peerreview.html Last modified: