PDF(Pubmed)
Abstract：
UNASSIGNED: The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Surprisingly little has been done to calibrate the public\'s evaluation of preprints and their contents. The PRECHECK project aimed to generate a tool to teach and guide scientifically literate non-experts to critically evaluate preprints, on COVID-19 and beyond.
UNASSIGNED: To create a checklist, we applied a four-step procedure consisting of an initial internal review, an external review by a pool of experts (methodologists, meta-researchers/experts on preprints, journal editors, and science journalists), a final internal review, and a Preliminary implementation stage. For the external review step, experts rated the relevance of each element of the checklist on five-point Likert scales, and provided written feedback. After each internal review round, we applied the checklist on a small set of high-quality preprints from an online list of milestone research works on COVID-19 and low-quality preprints, which were eventually retracted, to verify whether the checklist can discriminate between the two categories.
UNASSIGNED: At the external review step, 26 of the 54 contacted experts responded. The final checklist contained four elements (Research question, study type, transparency and integrity, and limitations), with \'superficial\' and \'deep\' evaluation levels. When using both levels, the checklist was effective at discriminating a small set of high- and low-quality preprints. Its usability for assessment and discussion of preprints was confirmed in workshops with Bachelors students in Psychology and Medicine, and science journalists.
UNASSIGNED: We created a simple, easy-to-use tool for helping scientifically literate non-experts navigate preprints with a critical mind and facilitate discussions within, for example, a beginner-level lecture on research methods. We believe that our checklist has potential to help guide decisions about the quality of preprints on COVID-19 in our target audience and that this extends beyond COVID-19.
UNASSIGNED: To create a checklist, we applied a four-step procedure consisting of an initial internal review, an external review by a pool of experts (methodologists, meta-researchers/experts on preprints, journal editors, and science journalists), a final internal review, and a Preliminary implementation stage. For the external review step, experts rated the relevance of each element of the checklist on five-point Likert scales, and provided written feedback. After each internal review round, we applied the checklist on a small set of high-quality preprints from an online list of milestone research works on COVID-19 and low-quality preprints, which were eventually retracted, to verify whether the checklist can discriminate between the two categories.
UNASSIGNED: At the external review step, 26 of the 54 contacted experts responded. The final checklist contained four elements (Research question, study type, transparency and integrity, and limitations), with \'superficial\' and \'deep\' evaluation levels. When using both levels, the checklist was effective at discriminating a small set of high- and low-quality preprints. Its usability for assessment and discussion of preprints was confirmed in workshops with Bachelors students in Psychology and Medicine, and science journalists.
UNASSIGNED: We created a simple, easy-to-use tool for helping scientifically literate non-experts navigate preprints with a critical mind and facilitate discussions within, for example, a beginner-level lecture on research methods. We believe that our checklist has potential to help guide decisions about the quality of preprints on COVID-19 in our target audience and that this extends beyond COVID-19.
摘要：
■应谨慎考虑COVID-19预印品的质量，因为它们的内容会影响公共政策。令人惊讶的是，在校准公众对预印本及其内容的评估方面几乎没有做任何事情。PRECECK项目旨在生成一种工具来教授和指导具有科学素养的非专家批判性地评估预印本，在COVID-19及以后。
■要创建清单，我们采用了一个由初步内部审查组成的四步程序，由一批专家进行的外部审查(方法学家，预印本的元研究人员/专家，期刊编辑,和科学记者)，最后的内部审查,初步实施阶段。对于外部审核步骤，专家以五点李克特量表对清单中每个元素的相关性进行了评估，并提供书面反馈。在每一轮内部审查之后，我们将清单应用于一小套高质量的预印本，这些预印本来自COVID-19的里程碑研究工作和低质量的预印本的在线列表，最终被收回，以验证清单是否可以区分这两个类别。
■在外部审查步骤中，接触的54位专家中有26位做出了回应。最终的检查表包含四个要素(研究问题，研究类型,透明度和完整性，和限制)，具有\'表面\'和\'深度\'评估级别。当使用这两个级别时，检查表有效地区分了一小部分高质量和低质量的预印本。在与心理学和医学学士学位学生的研讨会上证实了其评估和讨论预印本的可用性，科学记者。
■我们创建了一个简单的，易于使用的工具，用于帮助具有科学素养的非专家以批判性思维浏览预印本，并促进内部讨论，例如,关于研究方法的初级讲座。我们认为，我们的清单有可能帮助指导我们目标受众关于COVID-19预印本质量的决策，这超出了COVID-19。
■要创建清单，我们采用了一个由初步内部审查组成的四步程序，由一批专家进行的外部审查(方法学家，预印本的元研究人员/专家，期刊编辑,和科学记者)，最后的内部审查,初步实施阶段。对于外部审核步骤，专家以五点李克特量表对清单中每个元素的相关性进行了评估，并提供书面反馈。在每一轮内部审查之后，我们将清单应用于一小套高质量的预印本，这些预印本来自COVID-19的里程碑研究工作和低质量的预印本的在线列表，最终被收回，以验证清单是否可以区分这两个类别。
■在外部审查步骤中，接触的54位专家中有26位做出了回应。最终的检查表包含四个要素(研究问题，研究类型,透明度和完整性，和限制)，具有\'表面\'和\'深度\'评估级别。当使用这两个级别时，检查表有效地区分了一小部分高质量和低质量的预印本。在与心理学和医学学士学位学生的研讨会上证实了其评估和讨论预印本的可用性，科学记者。
■我们创建了一个简单的，易于使用的工具，用于帮助具有科学素养的非专家以批判性思维浏览预印本，并促进内部讨论，例如,关于研究方法的初级讲座。我们认为，我们的清单有可能帮助指导我们目标受众关于COVID-19预印本质量的决策，这超出了COVID-19。
