Advertisement

Peer Review and Wilderness & Environmental Medicine

      Peer review is a linchpin component of science and science communication. It is probably most recognized in the publication of research manuscripts. Authors prepare reports of their scientific efforts and submit them to a peer-reviewed journal thought to be appropriate for the content. Editorial staff assign these reports to subject matter expert reviewers who evaluate propriety and provide thoughtful comments to help authors improve manuscripts where needed.
      The procedure is compelling, but there are challenges. Foremost is the fact that the source data used to develop the manuscript are rarely made available to reviewers: Reviewers must judge the work without seeing the heart of it. This can lead to serious problems, but it is the mundane issues that arise more frequently. On the reviewer side, the best subject matter expert may not be available to review a given manuscript, or sufficient time may not be allotted to complete a comprehensive review. On the reviewee side, substandard efforts to revise even the most well-reviewed manuscript can lead to serial rereview, and fatigue on one or both sides can negatively affect the final product.
      Finding the right expert with the time and interest to provide an excellent review is a burden that editorial teams will likely always have to manage. The situation does improve, however, when the obligations and process of peer review are widely understood. It is dismaying to know that even with the critical importance of peer review, many individuals have little formal exposure to it in their training years. Their first experience may come only after they become working professionals.
      The quality of both submissions and reviews is improved with practice and understanding. Students can be engaged in an efficient manner by having them conduct what I call shadow reviews. They are given a copy of a manuscript that a mentor will review, and with a modest amount of guidance, they then complete a written review independent of the mentor’s effort. The two are then compared, giving the student insights into the finer points of the effort, giving the mentor additional motivation to put together a top notch review, and often uncovering at least a small number of points to add to the reviewer’s comments. Repeating the process allows students to develop their capabilities. It is worth noting that the first shadow review will often be painful to both student and mentor. The value will be reinforced for both, however, as subsequent efforts advance.
      There are 2 core elements to any manuscript review: subject matter expertise and process. Subject matter expertise is gained through professional growth, but a solid foundation for the review process can be established independently to improve the quality of any review. Stated differently, even a person unschooled in a specific subject matter can provide meaningful reviews when the structural elements are understood. Manuscripts frequently suffer from poor organization, often with results and methods or discussion and results comingled. One has to understand the normal structure, not the science, of the research paper to evaluate the construction. Similarly, referencing and logical disconnections are often issues in manuscripts, and these can be at least partially evaluated without subject matter expertise.
      Written shadow reviews have greater educational value than verbal reviews. Trainees learn more quickly to select their words carefully for clarity and diplomacy. The former is important since the authors cannot easily ask for further explanation. The latter is critical since a comment may well be wrong, and a polite inquiry is less likely to raise the ire of authors. Learning the range of phrasing is important, from explicit instruction when confidence is high to soft query when confidence is low. The broader benefit for trainees is that critical reading and technical writing abilities will improve, increasing the collective value.
      The schedule of reviewing and, by extension, of teaching through shadow reviews, can be inconstant, but regular opportunities can be created. Many journals retain rosters of persons willing to serve as reviewers. Candidates generally provide self-defined lists of subject matter expertise to focus invitations. Registering one’s preferences and skills with topical journals can bring invitations to review. Ultimately, participation benefits all—authors, trainees, the journal, and the community.
      Recognizing the importance of peer review, Wilderness & Environmental Medicine developed an award program for excellence in peer review in 2010. These awards recognize and reinforce high quality efforts. Reviewers are graded on 4 parameters: quality, consistency, commitment, and earned scores. The highest ranks for quality of reviews are given to those that are clear, comprehensive, and well-organized in an actionable form that makes it easy for authors to respond. They must also be delivered in a tone that is respectful of authors. It is important to remember that authors are at a disadvantage in the review process. They may be hesitant to ask questions or argue if they feel it might negatively affect the reviewer’s recommendation. Respect and politeness are important to help to defuse the situation.
      The consistency measure looks for agreement of tone between comments and scores shared confidentially with the editor and comments to authors. Although there should be no statement of the reviewer’s recommendation in the comments to authors and certainly no disrespect shown, reviewers should provide guidance to authors that is compatible with their recommendation to editors. Additional diplomacy is required in addressing authors, but the comments made should not mislead authors. For example, a reviewer should not be effusively positive in comments to authors and then recommend rejection of the manuscript.
      The commitment measure evaluates contribution to the process. Strong performance in this category is represented by a high frequency of informative reviews, a high rate of accepting invitations to review, and a high rate of requesting to see revised manuscripts. The willingness to work with authors through iterative efforts is important to achieve the best results.
      The final metric of earned scores is the grading of reviewer efforts by section editors or associate editors. Each review can be evaluated for its overall value to the authors and the goals of high-quality peer review. Strong performance is indicated by a consistent pattern in the good to excellent range. The grading scale is 90 to 100 (excellent); 80 to 89 (good); 70 to 79 (average); 60 to 69 (below average); and ≤59 (poor).
      A total of 29 individuals have been recognized as outstanding peer reviewers by Wilderness & Environmental Medicine between 2010 and 2016 (see http://wms.org/about/awards.asp). Please join us in promoting the highest quality of peer review, creating opportunities to involve and educate trainees, and potentially in being recognized for your abilities and contributions. Additional resources to prepare for reviewing responsibilities can be found in our Reviewer Toolkit, available at http://wms.org/research.