简介:重复性是研究的核心租户。我们旨在综合有关可重复性的文献,并描述其流行病学特征,包括如何定义和评估再现性。我们还旨在确定和比较不同领域的可重复性估计值。
方法:我们进行了范围审查,以确定2018-2019年间发表的经济学英语语言复制研究,教育,心理学,健康科学和生物医学。我们搜查了Medline,Embase,PsycINFO,护理和相关健康文献的累积指数-CINAHL,通过EBSCOHost的教育来源,ERIC,EconPapers,国际社会科学参考书目(IBSS),还有EconLit.根据我们的纳入标准,对检索到的文件进行一式两份的筛选。我们提取了出版年份,作者数量,通讯作者的所属国家,以及这项研究是否得到资助。对于单个复制研究,我们记录了是否使用了复制研究的注册方案,复制团队和原作者之间是否有联系,使用了什么研究设计,以及主要结果是什么。最后,我们记录了作者如何定义可重复性,以及根据此定义评估的研究是否成功复制。提取由单个审阅者完成,并且质量由第二审阅者控制。
结果:我们的搜索确定了11,224个独特的文档,其中47人被纳入本次审查。大多数研究与心理学(48.6%)或健康科学(23.7%)有关。在这47份文件中,36描述了单个再现性研究,而其余11在同一篇论文中报道了至少两个再现性研究。不到一半的研究涉及注册协议。重复性成功的定义存在差异。总的来说,在47份文件中,共报告了177项研究。根据每个研究的作者使用的定义,177项研究中有95项(53.7%)重复。
结论:本研究概述了五个学科的研究,这些学科明确地重复了以前的研究。这样的可重复性研究非常稀缺,成功复制的研究的定义是模糊的,再现率总体适中。
资金:这项工作没有收到外部资金。
Reproducibility is a central tenant of research. We aimed to synthesize the literature on reproducibility and describe its epidemiological characteristics, including how reproducibility is defined and assessed. We also aimed to determine and compare estimates for reproducibility across different fields.
We conducted a scoping
review to identify English language replication studies published between 2018 and 2019 in economics, education, psychology, health sciences, and biomedicine. We searched Medline, Embase, PsycINFO, Cumulative Index of Nursing and Allied Health Literature - CINAHL, Education Source via EBSCOHost, ERIC, EconPapers, International Bibliography of the Social Sciences (IBSS), and EconLit. Documents retrieved were screened in duplicate against our inclusion criteria. We extracted year of publication, number of authors, country of affiliation of the corresponding author, and whether the study was funded. For the individual replication studies, we recorded whether a registered protocol for the replication study was used, whether there was contact between the reproducing team and the original authors, what study design was used, and what the primary outcome was. Finally, we recorded how reproducibilty was defined by the authors, and whether the assessed study(ies) successfully reproduced based on this definition. Extraction was done by a single reviewer and quality controlled by a second reviewer.
Our search identified 11,224 unique documents, of which 47 were included in this
review. Most studies were related to either psychology (48.6%) or health sciences (23.7%). Among these 47 documents, 36 described a single reproducibility study while the remaining 11 reported at least two reproducibility studies in the same paper. Less than the half of the studies referred to a registered protocol. There was variability in the definitions of reproduciblity success. In total, across the 47 documents 177 studies were reported. Based on the definition used by the author of each study, 95 of 177 (53.7%) studies reproduced.
This study gives an overview of research across five disciplines that explicitly set out to reproduce previous research. Such reproducibility studies are extremely scarce, the definition of a successfully reproduced study is ambiguous, and the reproducibility rate is overall modest.
No external funding was received for this work.