%0 Journal Article
%T Scientists' perception of pilot study quality was influenced by statistical significance and study design.
%A von Klinggraeff L
%A Burkart S
%A Pfledderer CD
%A Saba Nishat MN
%A Armstrong B
%A Weaver RG
%A McLain AC
%A Beets MW
%J J Clin Epidemiol
%V 159
%N 0
%D 2023 07 20
%M 37217107
%F 7.407
%R 10.1016/j.jclinepi.2023.05.011
%X Preliminary studies play a key role in developing large-scale interventions but may be held to higher or lower scientific standards during the peer review process because of their preliminary study status.<BR>Abstracts from 5 published obesity prevention preliminary studies were systematically modified to generate 16 variations of each abstract. Variations differed by 4 factors: sample size (n = 20 vs. n = 150), statistical significance (P < 0.05 vs. P > 0.05), study design (single group vs. randomized 2 groups), and preliminary study status (presence/absence of pilot language). Using an online survey, behavioral scientists were provided with a randomly selected variation of each of the 5 abstracts and blinded to the existence of other variations. Respondents rated each abstract on aspects of study quality.<BR>Behavioral scientists (n = 271, 79.7% female, median age 34 years) completed 1,355 abstract ratings. Preliminary study status was not associated with perceived study quality. Statistically significant effects were rated as more scientifically significant, rigorous, innovative, clearly written, warranted further testing, and had more meaningful results. Randomized designs were rated as more rigorous, innovative, and meaningful.<BR>Findings suggest reviewers place a greater value on statistically significant findings and randomized control design and may overlook other important study characteristics.