PDF(Pubmed)
Abstract：
Preliminary studies play a key role in developing large-scale interventions but may be held to higher or lower scientific standards during the peer review process because of their preliminary study status.
Abstracts from 5 published obesity prevention preliminary studies were systematically modified to generate 16 variations of each abstract. Variations differed by 4 factors: sample size (n = 20 vs. n = 150), statistical significance (P < 0.05 vs. P > 0.05), study design (single group vs. randomized 2 groups), and preliminary study status (presence/absence of pilot language). Using an online survey, behavioral scientists were provided with a randomly selected variation of each of the 5 abstracts and blinded to the existence of other variations. Respondents rated each abstract on aspects of study quality.
Behavioral scientists (n = 271, 79.7% female, median age 34 years) completed 1,355 abstract ratings. Preliminary study status was not associated with perceived study quality. Statistically significant effects were rated as more scientifically significant, rigorous, innovative, clearly written, warranted further testing, and had more meaningful results. Randomized designs were rated as more rigorous, innovative, and meaningful.
Findings suggest reviewers place a greater value on statistically significant findings and randomized control design and may overlook other important study characteristics.
Abstracts from 5 published obesity prevention preliminary studies were systematically modified to generate 16 variations of each abstract. Variations differed by 4 factors: sample size (n = 20 vs. n = 150), statistical significance (P < 0.05 vs. P > 0.05), study design (single group vs. randomized 2 groups), and preliminary study status (presence/absence of pilot language). Using an online survey, behavioral scientists were provided with a randomly selected variation of each of the 5 abstracts and blinded to the existence of other variations. Respondents rated each abstract on aspects of study quality.
Behavioral scientists (n = 271, 79.7% female, median age 34 years) completed 1,355 abstract ratings. Preliminary study status was not associated with perceived study quality. Statistically significant effects were rated as more scientifically significant, rigorous, innovative, clearly written, warranted further testing, and had more meaningful results. Randomized designs were rated as more rigorous, innovative, and meaningful.
Findings suggest reviewers place a greater value on statistically significant findings and randomized control design and may overlook other important study characteristics.
摘要：
目的：初步研究在开发大规模干预措施方面发挥着关键作用，但由于其初步研究状态，在同行评审过程中可能会保持较高或较低的科学标准。
方法：对已发表的5项肥胖预防初步研究的摘要进行了系统修改，以生成每个摘要的16种变体。差异有四个因素：样本量(n=20与n=150),统计学意义（P<0.05vs.P>.05)，研究设计(单组vs.随机两组)，和初步研究状况(是否存在/不存在试点语言)。使用在线调查，为行为科学家提供了五个摘要中每个摘要的随机选择变体，并且不知道其他变体的存在。受访者在研究质量方面对每个摘要进行了评分。
结果：行为科学家（n=271，79.7％为女性，中位年龄34岁）完成了1,355个抽象评级。初步研究状态与感知的研究质量无关。统计上显著的效果被评为更科学显著，严谨,创新,写得很清楚,保证进一步的测试，结果更有意义。随机设计被评为更严格，创新,和有意义的。
结论：研究结果表明，研究者更重视统计学意义的研究结果和随机对照设计，可能忽略了其他重要的研究特征。
方法：对已发表的5项肥胖预防初步研究的摘要进行了系统修改，以生成每个摘要的16种变体。差异有四个因素：样本量(n=20与n=150),统计学意义（P<0.05vs.P>.05)，研究设计(单组vs.随机两组)，和初步研究状况(是否存在/不存在试点语言)。使用在线调查，为行为科学家提供了五个摘要中每个摘要的随机选择变体，并且不知道其他变体的存在。受访者在研究质量方面对每个摘要进行了评分。
结果：行为科学家（n=271，79.7％为女性，中位年龄34岁）完成了1,355个抽象评级。初步研究状态与感知的研究质量无关。统计上显著的效果被评为更科学显著，严谨,创新,写得很清楚,保证进一步的测试，结果更有意义。随机设计被评为更严格，创新,和有意义的。
结论：研究结果表明，研究者更重视统计学意义的研究结果和随机对照设计，可能忽略了其他重要的研究特征。
