背景:数字医疗保健应用程序,包括数字疗法,有可能通过克服传统的基于设施的医学治疗的局限性来增加可及性并改善患者的参与度。然而,目前还没有能够定量衡量长期参与的既定工具。
目的:本研究旨在评估长期使用的商业健康管理应用程序中现有的参与度指数(EI),并将其与新开发的EI进行比较。
方法:参与者从参加一项随机对照试验的癌症幸存者中招募,该试验评估了移动健康应用程序对康复的影响。在这些病人中,240人被纳入研究,并随机分配到Noom应用程序(NoomInc)。将新开发的EI与现有的EI进行了比较,并进行了长期使用分析。此外,新的EI是根据网络矩阵访问者指数的适应性测量进行评估的,专注于点击深度,最近,和忠诚度指数。
结果:新开发的EI模型在根据3至6个月的EI预测6至9个月的EI方面优于现有的EI模型。现有模型的均方误差为0.096,均方根误差为0.310,R2为0.053。同时,新开发的EI模型显示出改进的性能,最好的一个实现的均方误差为0.025,均方根误差为0.157,R2为0.610。现有的EI表现出显著的关联:点击深度指数(风险比[HR]0.49,95%CI0.29-0.84;P<.001)和忠诚度指数(HR0.17,95%CI0.09-0.31;P<.001)与提高生存率显著相关,而近期指数无显著相关性(HR1.30,95%CI1.70-2.42;P=.41)。在新的EI模型中,在应用程序的免费版本中提供菜单组合的EI产生了最有希望的结果。此外,其与忠诚度指数(HR0.32,95%CI0.16-0.62;P<.001)和新近度指数(HR0.47,95%CI0.30-0.75;P<.001)显著相关。
结论:新开发的EI模型在移动健康应用程序背景下的长期用户参与度和合规性预测方面优于现有模型。我们强调了日志数据的重要性,并提出了未来研究的途径,以解决EI的主观性,并纳入更广泛的指标进行综合评估。
BACKGROUND: Digital health care apps, including digital therapeutics, have the potential to increase accessibility and improve patient engagement by overcoming the limitations of traditional facility-based medical treatments. However, there are no established tools capable of quantitatively measuring long-term engagement at present.
OBJECTIVE: This study aimed to evaluate an existing engagement index (EI) in a commercial health management app for long-term use and compare it with a newly developed EI.
METHODS: Participants were recruited from cancer survivors enrolled in a randomized controlled trial that evaluated the impact of mobile health apps on recovery. Of these patients, 240 were included in the study and randomly assigned to the Noom app (Noom Inc). The newly developed EI was compared with the existing EI, and a long-term use analysis was conducted. Furthermore, the new EI was evaluated based on adapted measurements from the Web Matrix Visitor Index, focusing on click depth, recency, and loyalty indices.
RESULTS: The newly developed EI model outperformed the existing EI model in terms of predicting EI of a 6- to 9-month period based on the EI of a 3- to 6-month period. The existing model had a mean squared error of 0.096, a root mean squared error of 0.310, and an R2 of 0.053. Meanwhile, the newly developed EI models showed improved performance, with the best one achieving a mean squared error of 0.025, root mean squared error of 0.157, and R2 of 0.610. The existing EI exhibited significant associations: the click depth index (hazard ratio [HR] 0.49, 95% CI 0.29-0.84; P<.001) and loyalty index (HR 0.17, 95% CI 0.09-0.31; P<.001) were significantly associated with improved survival, whereas the recency index exhibited no significant association (HR 1.30, 95% CI 1.70-2.42; P=.41). Among the new EI models, the EI with a menu combination of menus available in the app\'s free version yielded the most promising result. Furthermore, it exhibited significant associations with the loyalty index (HR 0.32, 95% CI 0.16-0.62; P<.001) and the recency index (HR 0.47, 95% CI 0.30-0.75; P<.001).
CONCLUSIONS: The newly developed EI model outperformed the existing model in terms of the prediction of long-term user engagement and compliance in a mobile health app context. We emphasized the importance of log data and suggested avenues for future research to address the subjectivity of the EI and incorporate a broader range of indices for comprehensive evaluation.