Scientific journals are supposed to present meaningful reports of current research to educate readers on avenues explored, knowledge gained, and missteps best avoided in future investigations. Problems can arise, though, when external factors create a sense of urgency.
I have been highly critical of submissions making reference to COVID-19 in the first months following the outbreak. It is not because the topic is unimportant—quite the opposite, it is that the importance demands the same thoughtful and measured analyzes before public release that we expect for all work. The drive to be the first to report on important topics is powerful—often with some altruism to benefit the world, but also often with the added seduction of professional recognition, improved access to grant funding, and even simple competitiveness.
The problem is that we cannot afford to get it wrong on the big topics. Positions should change as new data become available and understanding evolves, but these changes are very different from the whiplash-inducing jumps that are more likely to be seen with the “rush to judgment” form of reporting.
While important findings should be shared rapidly, there should be a pause—the rationality check—before proceeding. There are 4 parts to the basic check: confirmation of the validity of the data, confirmation of the soundness of the interpretation, confirmation of the justification to discount alternate interpretations, and the willingness to go on public trial.
Hesitation over any part of the rationality check should delay manuscript submission. Peer review is a well-intentioned process to evaluate and aid manuscripts, but it is at best an imperfect safety net. Believing that reviewers represent the only gauntlet to be run is both reckless and foolish. Motivated reviewers will look for points of weakness and failure, but they are not taking on the role or responsibilities of coauthors. Reviewers will often catch flaws, but they may miss some, even critical ones. The responsibility for the final product remains with the authors, and the public trial begins when the paper is released to the wild; exposed to countless intelligent, insightful, and motivated minds critically evaluating the work.
Two high-profile papers related to COVID-19 were published in May and retracted in June 2020:
RETRACTED: Mehra MR, Desai SS, Ruschitzka F, Patel AN. Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet. 2020. https://doi.org/10.1016/S0140-6736(20)31180-6.
RETRACTED: Mehra MR, Desai SS, Kuy SR, Henry TD, Patel AN. Cardiovascular disease, drug therapy, and mortality in COVID-19. N Engl J Med. https://doi.org/10.1056/NEJMoa2007621.
There were comments after the retractions that these papers serve as examples of a broken peer review system, but this judgment is unfair. The peer review process used by the journals was not successful in these cases, but it was postpublication peer review that brought these papers down. It is an engaged readership that provides the broadest oversight for which authors must be prepared.
I wrote in my last editorial about the need to submit to and work for the most credible journals in our fields. I am going to add a requirement that manuscripts should only be submitted to journals that promote active discussion of published work. Issues in published work should be flagged through letters to the editor that point out deficiencies in a clear and respectful way. The appropriate editorial course is to welcome letters, provide authors with the opportunity to rebut them, and to be open to appropriate action to improve or correct the publication record.
Questions can be raised over the most appropriate course for a retraction—whether authors should be allowed to have a voice in the narrative of the decision to retract or not—but both of the journals listed in this case get credit for taking action.
Papers that deserve retraction should be retracted as soon as possible. Journals that discourage active discussion may be slower to act, hoping to be able to avoid taking action. In reality, the energy to force retraction through external pressure is likely only to be seen with high-profile reports. Community recognition of irresponsible practices can, though, still exact a price. All authors associated with a discredited journal may be tainted, even when their work is credible.
The decision to take the easy route of publishing in “pay-to-play” journals may seem like an acceptable choice, particularly if a manuscript has been rejected by more desirable journals, but the long-term risk to professional credibility arising from association with journals known for questionable practices must be considered.
Retracting papers is a messy business. “Retracted” may be watermarked across the pages or in the header section of pdf documents under the control of a journal or publisher, and retraction letters may be linked to the papers in search engine output, but a bad paper is like embarrassing photos from our youth: there is no way to reliably recall hard or digital copies once they have been circulated. While fully informed and competent critical readers should be able to handle retracted material appropriately, copies can still make their way to less savvy readers who perpetuate both unintentional and intentional misuse.
Legitimate cases of unintentional error that lead to retraction should be addressed as directly as possible. Mistakes are forgivable; it is the attempt to avoid acknowledging them that is far more likely to lead to long-term damage. The reality is that while predatory journals make it easier for invalid material to be published, every journal can receive manuscripts that are improperly developed, fatally flawed, or completely fraudulent.
Authors who are tempted to feel frustrated with what can seem like micromanagement in the review and revision process should remember that the back-and-forth can help to improve the product, evaluate the efforts of authors, and sometimes uncover issues that should affect publishability. Documentation of co-author contribution is intended to ensure that appropriate responsibility is taken.
The best strategy is to not let undeserving papers be published in the first place. This is an ongoing challenge that relies on the best combined efforts of researchers, research institutions, authors, reviewers, editors, and publication management tools. The checks and balances must be fully engaged to ensure that only appropriate material moves forward.
Readers should be invited to submit well-reasoned criticisms of content they believe has been published inappropriately. Letters and rebuttals can address or clarify shortcomings or perceived shortcomings. If problems rise to the level of fatal flaws, retraction is an appropriate course.
An expedited review process, well intended though it may be in times of crisis or critical need, can easily backfire. Turnaround times can sometimes be shortened in a limited number of cases without sacrificing rigor, but the sense of urgency can also result in shortcomings being overlooked. There are reasons for the existing timelines, and efforts to compress the various stages can adversely affect the final product.
One of the biggest risks to quality control is the release of “preprint” manuscripts—those that have not completed peer review. While it is true that preprints allow the public to see reports faster, the quality and even the publishability of such works is in no way assured. Even the similarity of the name with another standard of peer reviewed publications—preprint vs reprint—is likely to confuse some readers. The hazard is even greater when preprints are laid out in the style of regular articles, almost guaranteeing that they will be confused with peer-reviewed publications by some readers. Although the practice of releasing preprints may be well intended, it is fundamentally in conflict with the peer review process.
Our goal must be to maintain high standards and to do so transparently, informing authors and readers of expectations to instill confidence in the process and to actively engage in evaluation of the product. Letters to the editor are encouraged to advance the discussion of all work that we publish. We will also employ electronic publication in advance of print, but never of “preprint” material that has not successfully navigated peer review.
Article info
Publication history
Published online: August 18, 2020
Identification
Copyright
© 2020 Wilderness Medical Society. Published by Elsevier Ltd. All rights reserved.
