Abstract：
Symptom measurement in psychiatric research increasingly uses digitized self-report inventories and is turning to crowdsourcing platforms for recruitment, e.g., Amazon Mechanical Turk (mTurk). The impact of digitizing pencil-and-paper inventories on the psychometric properties is underexplored in mental health research. Against this background, numerous studies report high prevalence estimates of psychiatric symptoms in mTurk samples. Here we develop a framework to evaluate the online implementation of psychiatric symptom inventories relative to two domains, that is, the adherence to (i) validated scoring and (ii) standardized administration. We apply this new framework to the online use of the Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder-7 (GAD-7), and Alcohol Use Disorder Identification Test (AUDIT). Our systematic review of the literature identified 36 implementations of these three inventories on mTurk across 27 publications. We also evaluated methodological approaches to enhance data quality, e.g., the use of bot detection and attention check items. Of the 36 implementations, 23 reported the applied diagnostic scoring criteria and only 18 reported the specified symptom timeframe. None of the 36 implementations reported adaptations made in their digitization of the inventories. While recent reports attribute higher rates of mood, anxiety, and alcohol use disorders on mTurk to data quality, our findings indicate that this inflation may also relate to the assessment methods. We provide recommendations to enhance both data quality and fidelity to validated administration and scoring methods.
摘要：
精神病学研究中的症状测量越来越多地使用数字化的自我报告清单，并转向众包平台进行招聘，例如，亚马逊机械土耳其人(mTurk)。在心理健康研究中，未充分研究数字化铅笔和纸库存对心理测量特性的影响。在这种背景下,许多研究报告了mTurk样本中精神症状的高患病率估计值。在这里，我们开发了一个框架来评估精神症状清单相对于两个领域的在线实施情况，也就是说,对(i)验证评分和(ii)标准化给药的依从性.我们将这个新框架应用于在线使用患者健康问卷-9(PHQ-9)，广义焦虑症-7（GAD-7），和酒精使用障碍识别测试（AUDIT）。我们对文献的系统回顾在27种出版物中确定了mTurk上这三种清单的36种实现。我们还评估了提高数据质量的方法学方法，例如，使用bot检测和注意检查项目。在36个实现中，23报告了应用的诊断评分标准，只有18报告了指定的症状时间范围。36个实施例中没有一个报告在清单数字化方面进行了调整。虽然最近的报告归因于较高的情绪率，焦虑,以及mTurk上的酒精使用障碍对数据质量的影响，我们的研究结果表明，这种通货膨胀也可能与评估方法有关。我们提供建议，以提高数据质量和保真度，以验证管理和评分方法。
