When deciding which submissions should be peer reviewed, eLife editors consider whether they will be able to find high-quality reviewers, and whether the reviews will be valuable to the scientific community.

UNASSIGNED: The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Surprisingly little has been done to calibrate the public\'s evaluation of preprints and their contents. The PRECHECK project aimed to generate a tool to teach and guide scientifically literate non-experts to critically evaluate preprints, on COVID-19 and beyond.
UNASSIGNED: To create a checklist, we applied a four-step procedure consisting of an initial internal review, an external review by a pool of experts (methodologists, meta-researchers/experts on preprints, journal editors, and science journalists), a final internal review, and a Preliminary implementation stage. For the external review step, experts rated the relevance of each element of the checklist on five-point Likert scales, and provided written feedback. After each internal review round, we applied the checklist on a small set of high-quality preprints from an online list of milestone research works on COVID-19 and low-quality preprints, which were eventually retracted, to verify whether the checklist can discriminate between the two categories.
UNASSIGNED: At the external review step, 26 of the 54 contacted experts responded. The final checklist contained four elements (Research question, study type, transparency and integrity, and limitations), with \'superficial\' and \'deep\' evaluation levels. When using both levels, the checklist was effective at discriminating a small set of high- and low-quality preprints. Its usability for assessment and discussion of preprints was confirmed in workshops with Bachelors students in Psychology and Medicine, and science journalists.
UNASSIGNED: We created a simple, easy-to-use tool for helping scientifically literate non-experts navigate preprints with a critical mind and facilitate discussions within, for example, a beginner-level lecture on research methods. We believe that our checklist has potential to help guide decisions about the quality of preprints on COVID-19 in our target audience and that this extends beyond COVID-19.

暂无摘要。

What happened when eLife decided to eliminate accept/reject decisions after peer review?

The process of peer review has been the gold standard for evaluating medical science, but significant pressures from the recent COVID-19 pandemic, new methods of communication, larger amounts of research, and an evolving publication landscape have placed significant pressures on this system. A task force convened by the American College of Cardiology identified the 5 most significant controversies associated with the current peer-review process: the effect of preprints, reviewer blinding, reviewer selection, reviewer incentivization, and publication of peer reviewer comments. Although specific solutions to these issues will vary, regardless of how scientific communication evolves, peer review must remain an essential process for ensuring scientific integrity, timely dissemination of information, and better patient care. In medicine, the peer-review process is crucial because harm can occur if poor-quality data or incorrect conclusions are published. With the dramatic increase in scientific publications and new methods of communication, high-quality peer review is more important now than ever.

Science journalists are uniquely positioned to increase the societal impact of open research outputs by contextualizing and communicating findings in ways that highlight their relevance and implications for non-specialist audiences. Yet, it is unclear to what degree journalists use open research outputs, such as open access publications or preprints, in their reporting; what factors motivate or constrain this use; and how the recent surge in openly available research seen during the COVID-19 pandemic has affected this. This article examines these questions through a review of relevant literature published from 2018 onwards-particularly literature relating to the COVID-19 pandemic-as well as seminal articles outside the search dates. We find that research that explicitly examines journalists\' engagement with open access publications or preprints is scarce, with existing literature mostly addressing the topic tangentially or as a secondary concern, rather than a primary focus. Still, the limited body of evidence points to several factors that may hamper journalists\' use of these outputs and thus warrant further exploration. These include an overreliance on traditional criteria for evaluating scientific quality; concerns about the trustworthiness of open research outputs; and challenges using and verifying the findings. We also find that, while the COVID-19 pandemic encouraged journalists to explore open research outputs such as preprints, the extent to which these explorations will become established journalistic practices remains unclear. Furthermore, we note that current research is overwhelmingly authored and focused on the Global North, and the United States specifically. We conclude with recommendations for future research that attend to issues of equity and diversity, and more explicitly examine the intersections of open access and science journalism.

暂无摘要。

The COVID-19 pandemic caused a rise in preprinting, triggered by the need for open and rapid dissemination of research outputs. We surveyed authors of COVID-19 preprints to learn about their experiences with preprinting their work and also with publishing their work in a peer-reviewed journal. Our research had the following objectives: 1. to learn about authors\' experiences with preprinting, their motivations, and future intentions; 2. to consider preprints in terms of their effectiveness in enabling authors to receive feedback on their work; 3. to compare the impact of feedback on preprints with the impact of comments of editors and reviewers on papers submitted to journals. In our survey, 78% of the new adopters of preprinting reported the intention to also preprint their future work. The boost in preprinting may therefore have a structural effect that will last after the pandemic, although future developments will also depend on other factors, including the broader growth in the adoption of open science practices. A total of 53% of the respondents reported that they had received feedback on their preprints. However, more than half of the feedback was received through \"closed\" channels-privately to the authors. This means that preprinting was a useful way to receive feedback on research, but the value of feedback could be increased further by facilitating and promoting \"open\" channels for preprint feedback. Almost a quarter of the feedback received by respondents consisted of detailed comments, showing the potential of preprint feedback to provide valuable comments on research. Respondents also reported that, compared to preprint feedback, journal peer review was more likely to lead to major changes to their work, suggesting that journal peer review provides significant added value compared to feedback received on preprints.

Peer-reviewed journals provide an invaluable but inadequate vehicle for scientific communication. Preprints are now an essential complement to peer-reviewed publications. Eschewing preprints will slow scientific progress and reduce the public health impact of epidemiologic research. The coronavirus disease 2019 (COVID-19) pandemic highlighted long-standing limitations of the peer-review process. Preprint servers, such as bioRxiv and medRxiv, served as crucial venues to rapidly disseminate research and provide detailed backup to sound-bite science that is often communicated through the popular press or social media. The major criticisms of preprints arise from an unjustified optimism about peer review. Peer review provides highly imperfect sorting and curation of research and only modest improvements in research conduct or presentation for most individual papers. The advantages of peer review come at the expense of months to years of delay in sharing research methods or results. For time-sensitive evidence, these delays can lead to important missteps and ill-advised policies. Even with research that is not intrinsically urgent, preprints expedite debate, expand engagement, and accelerate progress. The risk that poor-quality papers will have undue influence because they are posted on a preprint server is low. If epidemiology aims to deliver evidence relevant for public health, we need to embrace strategic uses of preprint servers.

In weighing the question of whether AJE should accept preprints that have received press coverage, we need to keep in mind 3 sets of interests: the public interest, the publisher\'s interest, and the author\'s interest. During public health emergencies, such as a pandemic, the author\'s interests (rapid communication of scientific findings to the public) are aligned with the public interest (learning about life-saving information as early as possible). However, the interests of different parties are not always aligned. In most cases, preprinted articles do not concern matters of life or death. Widespread dissemination of studies via preprint services conflicts with the journal editor\'s interest in delivering fresh, original content. Dissemination of study results prior to peer review can occasionally backfire and cause unintended harm if the findings turn out to be false.