The last Editor’s Note described peer review as a linchpin component of science and science communication and discussed strategies to train students in the process. In a perfect world, that would be enough: we would train the emerging professionals, community members would accept invitations to review only for the papers for which they had appropriate subject matter expertise, they would apply due diligence to comprehensively and objectively review every manuscript, and every paper accepted for publication would be fully vetted and sound.
The problem is that we do not work in a perfect world. Reviewers are sometimes too ambitious in describing their areas of expertise, which can trigger too many, and sometimes inappropriate, invitations to review. A well-meaning but inadequately equipped reviewer can miss critical issues in a manuscript. Invitees can sometimes opt for perfunctory reviews instead of declining them initially or when it becomes apparent that they have insufficient time for the effort. And some, hopefully rarely, may intentionally subvert the peer review system.
Numerous examples of peer review failures can be found in the literature: poor reviews, biased reviews, and faked reviews arranged by authors. These reports should not be interpreted as describing a failed system, but rather as a call to arms to protect what can be a very good system. Professionals giving generously of their time can help to ensure that what is published is fair, balanced, and understandable to most readers. Reviewers have the opportunity to contribute their insights into the historical record; being part of the greater good.
We ask authors to list persons who could serve as good reviewers and those with cause to be excluded. I share the concerns of others in the community that using these lists directly can have a negative impact on the review process. Some of the fraud that has been discovered involves authors setting up reviews done by themselves or collaborators. In part as a response to these concerns, reviewer recommendations are frequently not used as advertised. Instead, the recommended list might only be used to identify possible subject matter experts for future reviews or to exclude persons from invitation. Similarly, opposed reviewers may sometimes be identified as the best candidates for the job.
Our standard is to send manuscripts out for two independent reviews that are then evaluated by a section editor and/or the editor-in-chief. The section editor and/or editor-in-chief also review the manuscript. If the collective reviews appear to adequately evaluate the manuscript, the cycle is complete. If one or more of the reviews appear to be wanting, or if the recommendations are in marked conflict, additional reviews are solicited. Reviews that appear to be inadequate raise important flags that can affect future assignments.
When the review cycle is complete, the decision is made by the editor-in-chief and sent with all the reviews to the authors and each reviewer. Authors can begin work on revisions and engaged reviewers can see how their efforts compare to others on focus, insight, breadth, clarity, and comprehensiveness. The best reviewers will constantly refine their skill. Poor reviewers might improve their efforts or withdraw from the process, both being acceptable outcomes.
Compiled reviews provide an important record of the effort involved in the process. Comprehensive and constructive reviews do not guarantee propriety, but they can uncover important issues and test authors. The review process is a negotiation, almost always a compromise, but with a clear goal of producing the best product possible. Authors can struggle to address all of the reviewer comments in revision, and make a case to ignore or modify specific comments that they deem inappropriate. Major revisions that are made too easily can serve as a flag for additional questions. One of the limitations of the current peer review system is that reviewers typically evaluate only the manuscripts and not the source data. Vigilance is required for “too good to be true” manuscripts since fraud is on the continuum of possibilities.
The back-and-forth of the peer review and re-review process can be a frustration to authors, reviewers, and editors. The steps are necessary, though, to ensure that what is finally cleared for publication is material that we will be comfortable living with in the future. This is not to say that we may like, agree with, or even believe in the content of each paper, but that we have faith in the process to produce a product to be proud of.
Article info
Identification
Copyright
All rights reserved.
