When you review a manuscript, – Thank you! – how does a journal deal with your recommendation? Can a bad review really vote an author off the island? What happens when reviewers differ sharply? How many reviews are enough? Does someone actually read your full comments? These questions critically affect what you see in a journal.
Some journals, particularly newer ones offering open access, handle reviews with a point system. Gather enough accepts and you’re in, though negatives can kill you. In my view this is a cop-out for the editor. Why even bother to read the reviews, if all you have to do is tally up the points? A point system fails in two very significant ways: inconsistency of reviewers in assigning ratings, and, more importantly, inability to deal with genuine scientific controversy.
For reviews, the devil is in the details. I have seen a manuscript get rankings from accept to reject, yet all reviewer comments were almost the same. Some reviewers check “minor revisions” when those revisions are anything but minor. Others recommend rejection when all that’s needed is a good copy edit. Admittedly, these are extremes, but they happen often enough to matter. There is no substitute for a good editorial team carefully considering the reviewers’ comments.
Controversy is where editors earn their keep. A lot of good ideas started out controversial: evolution, plate tectonics, and climate change, to name a few. Bad ideas, too: cold fusion, anyone? Eminent scientists sometimes disagree emphatically, each genuinely feeling they are absolutely right and the other folks absolutely wrong. Ideas can take time to mature, and open discussion is a key process for science to reach a widely accepted opinion. Allow an absolute reviewer veto, and the next Origin of Species never gets published.
So, what to do when an author and reviewer are at loggerheads over some issue? First, we see what other reviewers say, maybe add more reviewers. Are both sides supporting their points with evidence and reason? Has the author resolved those comments that can be resolved? At some point, it becomes clear the debate needs to go to a larger community than the small group of reviewers and authors involved with a paper. (I like to use the term, “agree to disagree,” though the “agree” part sometimes is dicey.) We usually require a contested paper to at least acknowledge the existence of different opinions, and invite the dissenting reviewer to prepare a Discussion or their own opposing paper. Some debates are best held in an open forum
Publishing can be a messy process. I make around 500 editorial decisions a year. I know some are wrong; just don’t know which! The only thing you have the right to expect is that every decision will be made honestly with the facts at hand. And, yes, we at JAWRA read all those reviewer comments!