背景:与当面方法相比,使用远程方法评估数字干预可以相对方便和便宜地招募大量参与者。然而,根据参与者的自我报告远程进行研究,几乎没有验证,这对自动“机器人”和参与者的欺骗是开放的。
目的:本文使用了一项远程进行的酒精减少应用程序试验的案例研究,以强调和讨论(1)参与者欺骗影响远程研究试验的问题,并获得经济补偿;(2)严格的数据管理对检测和解决这些问题的重要性。
方法:我们从2020年7月至2022年3月在互联网上招募参与者,进行一项随机对照试验(n=5602),评估酒精减少应用的有效性。少喝。随访发生在3个时间点,提供经济补偿(最高36英镑[39.23美元])。地址认证和电话验证被用来检测两种欺骗:“机器人,“也就是说,在集群中生成的自动响应;以及手动参与者欺骗,也就是说,参与者提供虚假信息。
结果:在招募的前2个月中注册的1142名参与者中,其中75.6%(n=863)在数据筛选过程中被识别为机器人。因此,添加了验证码(全自动公共图灵测试,以告知计算机和人类分离),在这之后,没有更多的机器人被发现。在整个研究过程中都发生了手动参与者欺骗行为。在参加研究的5956名参与者(不包括机器人)中,298人(5%)被确定为假参与者。这一程度从2020年11月的110下降到2022年2月的可忽略不计的水平,包括几个月的0。下降发生在我们增加了注意力检查等进一步筛查问题之后,消除了社交媒体广告中经济补偿的突出地位,并增加了提供手机号码以进行身份验证的额外要求。
结论:数据管理协议对于在远程进行的试验中检测自动机器人和手动参与者欺骗是必要的。机器人和手动欺骗可以通过添加验证码来最小化,注意检查,提供身份验证电话号码的要求,而不是在社交媒体上突出宣传经济补偿。
背景:ISRCTN编号ISRCTN64052601;https://doi.org/10.1186/ISRCTN64052601。
Evaluating digital interventions using remote methods enables the recruitment of large numbers of participants relatively conveniently and cheaply compared with in-person methods. However, conducting research remotely based on participant self-report with little verification is open to automated \"bots\" and participant deception.
This paper uses a
case study of a remotely conducted trial of an alcohol reduction app to highlight and discuss (1) the issues with participant
deception affecting remote research trials with financial compensation; and (2) the importance of rigorous data management to detect and address these issues.
We recruited participants on the internet from July 2020 to March 2022 for a randomized controlled trial (n=5602) evaluating the effectiveness of an alcohol reduction app, Drink Less. Follow-up occurred at 3 time points, with financial compensation offered (up to £36 [US $39.23]). Address authentication and telephone verification were used to detect 2 kinds of
deception: \"bots,\" that is, automated responses generated in clusters; and manual participant deception, that is, participants providing false information.
Of the 1142 participants who enrolled in the first 2 months of recruitment, 75.6% (n=863) of them were identified as bots during data screening. As a result, a CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart) was added, and after this, no more bots were identified. Manual participant
deception occurred throughout the study. Of the 5956 participants (excluding bots) who enrolled in the study, 298 (5%) were identified as false participants. The extent of this decreased from 110 in November 2020, to a negligible level by February 2022 including a number of months with 0. The decline occurred after we added further screening questions such as attention checks, removed the prominence of financial compensation from social media advertising, and added an additional requirement to provide a mobile phone number for identity verification.
Data management protocols are necessary to detect automated bots and manual participant
deception in remotely conducted trials. Bots and manual deception can be minimized by adding a CAPTCHA, attention checks, a requirement to provide a phone number for identity verification, and not prominently advertising financial compensation on social media.
ISRCTN Number ISRCTN64052601; https://doi.org/10.1186/ISRCTN64052601.