PDF(Pubmed)
Abstract：
BACKGROUND: The ability to predict rheumatoid arthritis (RA) flares between clinic visits based on real-time, longitudinal patient-generated data could potentially allow for timely interventions to avoid disease worsening.
OBJECTIVE: This exploratory study aims to investigate the feasibility of using machine learning methods to classify self-reported RA flares based on a small data set of daily symptom data collected on a smartphone app.
METHODS: Daily symptoms and weekly flares reported on the Remote Monitoring of Rheumatoid Arthritis (REMORA) smartphone app from 20 patients with RA over 3 months were used. Predictors were several summary features of the daily symptom scores (eg, pain and fatigue) collected in the week leading up to the flare question. We fitted 3 binary classifiers: logistic regression with and without elastic net regularization, a random forest, and naive Bayes. Performance was evaluated according to the area under the curve (AUC) of the receiver operating characteristic curve. For the best-performing model, we considered sensitivity and specificity for different thresholds in order to illustrate different ways in which the predictive model could behave in a clinical setting.
RESULTS: The data comprised an average of 60.6 daily reports and 10.5 weekly reports per participant. Participants reported a median of 2 (IQR 0.75-4.25) flares each over a median follow-up time of 81 (IQR 79-82) days. AUCs were broadly similar between models, but logistic regression with elastic net regularization had the highest AUC of 0.82. At a cutoff requiring specificity to be 0.80, the corresponding sensitivity to detect flares was 0.60 for this model. The positive predictive value (PPV) in this population was 53%, and the negative predictive value (NPV) was 85%. Given the prevalence of flares, the best PPV achieved meant only around 2 of every 3 positive predictions were correct (PPV 0.65). By prioritizing a higher NPV, the model correctly predicted over 9 in every 10 non-flare weeks, but the accuracy of predicted flares fell to only 1 in 2 being correct (NPV and PPV of 0.92 and 0.51, respectively).
CONCLUSIONS: Predicting self-reported flares based on daily symptom scorings in the preceding week using machine learning methods was feasible. The observed predictive accuracy might improve as we obtain more data, and these exploratory results need to be validated in an external cohort. In the future, analysis of frequently collected patient-generated data may allow us to predict flares before they unfold, opening opportunities for just-in-time adaptative interventions. Depending on the nature and implication of an intervention, different cutoff values for an intervention decision need to be considered, as well as the level of predictive certainty required.
OBJECTIVE: This exploratory study aims to investigate the feasibility of using machine learning methods to classify self-reported RA flares based on a small data set of daily symptom data collected on a smartphone app.
METHODS: Daily symptoms and weekly flares reported on the Remote Monitoring of Rheumatoid Arthritis (REMORA) smartphone app from 20 patients with RA over 3 months were used. Predictors were several summary features of the daily symptom scores (eg, pain and fatigue) collected in the week leading up to the flare question. We fitted 3 binary classifiers: logistic regression with and without elastic net regularization, a random forest, and naive Bayes. Performance was evaluated according to the area under the curve (AUC) of the receiver operating characteristic curve. For the best-performing model, we considered sensitivity and specificity for different thresholds in order to illustrate different ways in which the predictive model could behave in a clinical setting.
RESULTS: The data comprised an average of 60.6 daily reports and 10.5 weekly reports per participant. Participants reported a median of 2 (IQR 0.75-4.25) flares each over a median follow-up time of 81 (IQR 79-82) days. AUCs were broadly similar between models, but logistic regression with elastic net regularization had the highest AUC of 0.82. At a cutoff requiring specificity to be 0.80, the corresponding sensitivity to detect flares was 0.60 for this model. The positive predictive value (PPV) in this population was 53%, and the negative predictive value (NPV) was 85%. Given the prevalence of flares, the best PPV achieved meant only around 2 of every 3 positive predictions were correct (PPV 0.65). By prioritizing a higher NPV, the model correctly predicted over 9 in every 10 non-flare weeks, but the accuracy of predicted flares fell to only 1 in 2 being correct (NPV and PPV of 0.92 and 0.51, respectively).
CONCLUSIONS: Predicting self-reported flares based on daily symptom scorings in the preceding week using machine learning methods was feasible. The observed predictive accuracy might improve as we obtain more data, and these exploratory results need to be validated in an external cohort. In the future, analysis of frequently collected patient-generated data may allow us to predict flares before they unfold, opening opportunities for just-in-time adaptative interventions. Depending on the nature and implication of an intervention, different cutoff values for an intervention decision need to be considered, as well as the level of predictive certainty required.
摘要：
背景：基于实时预测门诊就诊之间类风湿关节炎（RA）发作的能力，纵向患者产生的数据可能有助于及时进行干预,以避免疾病恶化.
目的：这项探索性研究旨在研究使用机器学习方法根据在智能手机应用程序上收集的每日症状数据的小数据集对自我报告的RA耀斑进行分类的可行性。
方法：使用远程监测类风湿关节炎（REMORA）智能手机应用程序报告的20名超过3个月的RA患者的每日症状和每周耀斑。预测因子是每日症状评分的几个汇总特征(例如，疼痛和疲劳)收集在引发耀斑问题的一周内。我们拟合了3个二元分类器：有和没有弹性网络正则化的逻辑回归，随机森林，天真的贝叶斯。根据接受者工作特征曲线的曲线下面积(AUC)评价性能。对于性能最好的模型，我们考虑了不同阈值的敏感性和特异性,以说明预测模型在临床环境中的不同表现方式.
结果：数据包括每位参与者平均60.6份每日报告和10.5份每周报告。参与者报告的中位随访时间为81天（IQR79-82天），每次发作的中位数为2（IQR0.75-4.25）。模型之间的AUC大致相似，但弹性网络正则化逻辑回归的AUC最高为0.82。在要求特异性为0.80的截止值下，该模型检测耀斑的相应灵敏度为0.60。该人群的阳性预测值（PPV）为53％，阴性预测值(NPV)为85%。鉴于耀斑的流行，获得的最佳PPV意味着每3个阳性预测中只有约2个是正确的(PPV0.65).通过优先考虑更高的净现值，该模型在每10个非耀斑周内正确预测了9个以上，但是预测耀斑的准确性下降到只有1/2是正确的（NPV和PPV分别为0.92和0.51）。
结论：使用机器学习方法根据前一周的每日症状评分预测自我报告的耀斑是可行的。随着我们获得更多数据，观察到的预测准确性可能会提高，这些探索性结果需要在外部队列中进行验证。在未来,分析频繁收集的患者生成的数据可能使我们能够在耀斑展开之前预测耀斑，为及时的适应性干预提供机会。根据干预的性质和含义，需要考虑干预决策的不同截止值，以及所需的预测确定性水平。
目的：这项探索性研究旨在研究使用机器学习方法根据在智能手机应用程序上收集的每日症状数据的小数据集对自我报告的RA耀斑进行分类的可行性。
方法：使用远程监测类风湿关节炎（REMORA）智能手机应用程序报告的20名超过3个月的RA患者的每日症状和每周耀斑。预测因子是每日症状评分的几个汇总特征(例如，疼痛和疲劳)收集在引发耀斑问题的一周内。我们拟合了3个二元分类器：有和没有弹性网络正则化的逻辑回归，随机森林，天真的贝叶斯。根据接受者工作特征曲线的曲线下面积(AUC)评价性能。对于性能最好的模型，我们考虑了不同阈值的敏感性和特异性,以说明预测模型在临床环境中的不同表现方式.
结果：数据包括每位参与者平均60.6份每日报告和10.5份每周报告。参与者报告的中位随访时间为81天（IQR79-82天），每次发作的中位数为2（IQR0.75-4.25）。模型之间的AUC大致相似，但弹性网络正则化逻辑回归的AUC最高为0.82。在要求特异性为0.80的截止值下，该模型检测耀斑的相应灵敏度为0.60。该人群的阳性预测值（PPV）为53％，阴性预测值(NPV)为85%。鉴于耀斑的流行，获得的最佳PPV意味着每3个阳性预测中只有约2个是正确的(PPV0.65).通过优先考虑更高的净现值，该模型在每10个非耀斑周内正确预测了9个以上，但是预测耀斑的准确性下降到只有1/2是正确的（NPV和PPV分别为0.92和0.51）。
结论：使用机器学习方法根据前一周的每日症状评分预测自我报告的耀斑是可行的。随着我们获得更多数据，观察到的预测准确性可能会提高，这些探索性结果需要在外部队列中进行验证。在未来,分析频繁收集的患者生成的数据可能使我们能够在耀斑展开之前预测耀斑，为及时的适应性干预提供机会。根据干预的性质和含义，需要考虑干预决策的不同截止值，以及所需的预测确定性水平。
