背景:数字糖尿病预防计划(dDPP)是有效的“数字处方”,但流失率和计划未完成。为了解决这个问题,我们开发了一种个性化的自动消息传递系统(PAMS),该系统利用SMS文本消息传递和数据集成到临床工作流程中,通过增强的患者-提供者沟通来提高dDPP参与度.初步数据显示阳性结果。然而,需要进一步调查,以确定如何根据用户的偏好优化PAMS等支持技术的定制,以提高其dDPP参与度。
目的:本研究评估利用机器学习(ML)开发dDPP用户的数字参与表型,并评估ML预测dDPP活动参与的准确性。这项研究将用于PAMS优化过程,通过结合参与度预测和数字表型来改善PAMS个性化。本研究旨在(1)证明使用dDPP用户收集的数据来构建预测参与度并有助于识别数字参与度表型的ML模型的可行性,(2)描述使用dDPP数据集开发ML模型的方法,并给出初步结果,和(3)基于ML模型输出提供有关用户剖析的初步数据。
方法:使用梯度增强森林模型,我们预测参与4个dDPP个人活动(体力活动,教训,社会活动,和称重)和一般活动(参与任何活动)基于应用程序中先前的短期和长期活动。接收器工作特性曲线下的面积,精确度-召回率曲线下的面积,和Brier得分指标决定了模型的性能。Shapley值反映了模型的特征重要性,并通过潜在的配置文件分析确定了哪些变量为用户提供了信息。
结果:我们使用每周和每日DPP数据集开发了2个模型(328,821和704,242条记录,分别),预测准确率超过90%。尽管两种模型都非常准确,每日模型更适合我们的研究计划,因为它预测了个人活动的每日变化,这对创造“数字表型”至关重要。“为了更好地理解对模型预测结果有贡献的变量,我们计算了两个模型的Shapley值,以确定对模型拟合贡献最大的特征;在过去7天中参与dDPP中的任何活动具有最大的预测能力.我们在与dDPP接触2周后(贝叶斯信息标准=-3222.46)对用户进行潜在配置文件分析,并确定了6个用户配置文件,包括那些参与度高的人,最小的参与,和减员。
结论:初步结果表明,应用具有预测能力的ML方法是定制和优化消息传递干预措施以支持患者参与和坚持数字处方的可接受机制。这些结果使我们能够在未来优化现有的消息传递平台,并将该方法扩展到其他临床领域。
背景:ClinicalTrials.govNCT04773834;https://www.clinicaltrials.gov/ct2/show/NCT04773834.
■RR2-10.2196/26750。
BACKGROUND: Digital diabetes prevention programs (dDPPs) are effective \"digital prescriptions\" but have high attrition rates and program noncompletion. To address this, we developed a personalized automatic messaging system (PAMS) that leverages SMS text messaging and data integration into clinical workflows to increase dDPP engagement via enhanced patient-provider communication. Preliminary data showed positive results. However, further investigation is needed to determine how to optimize the tailoring of support technology such as PAMS based on a user\'s preferences to boost their dDPP engagement.
OBJECTIVE: This study evaluates leveraging machine learning (ML) to develop digital engagement phenotypes of dDPP users and assess ML\'s accuracy in predicting engagement with dDPP activities. This research will be used in a PAMS optimization process to improve PAMS personalization by incorporating engagement prediction and digital phenotyping. This study aims (1) to prove the feasibility of using dDPP user-collected data to build an ML model that predicts engagement and contributes to identifying digital engagement phenotypes, (2) to describe methods for developing ML models with dDPP data sets and present preliminary results, and (3) to present preliminary data on user profiling based on ML model outputs.
METHODS: Using the gradient-boosted forest model, we predicted engagement in 4 dDPP individual activities (physical activity, lessons, social activity, and weigh-ins) and general activity (engagement in any activity) based on previous short- and long-term activity in the app. The area under the receiver operating characteristic curve, the area under the precision-recall curve, and the Brier score metrics determined the performance of the model. Shapley values reflected the feature importance of the models and determined what variables informed user profiling through latent profile analysis.
RESULTS: We developed 2 models using weekly and daily DPP data sets (328,821 and 704,242 records, respectively), which yielded predictive accuracies above 90%. Although both models were highly accurate, the daily model better fitted our research plan because it predicted daily changes in individual activities, which was crucial for creating the \"digital phenotypes.\" To better understand the variables contributing to the model predictor, we calculated the Shapley values for both models to identify the features with the highest contribution to model fit; engagement with any activity in the dDPP in the last 7 days had the most predictive power. We profiled users with latent profile analysis after 2 weeks of engagement (Bayesian information criterion=-3222.46) with the dDPP and identified 6 profiles of users, including those with high engagement, minimal engagement, and attrition.
CONCLUSIONS: Preliminary results demonstrate that applying ML methods with predicting power is an acceptable mechanism to tailor and optimize messaging interventions to support patient engagement and adherence to digital prescriptions. The results enable future optimization of our existing messaging platform and expansion of this methodology to other clinical domains.
BACKGROUND: ClinicalTrials.gov NCT04773834; https://www.clinicaltrials.gov/ct2/show/NCT04773834.
UNASSIGNED: RR2-10.2196/26750.