PDF(Pubmed)
Abstract：
BACKGROUND: Individualizing and optimizing treatment of relapsing-remitting multiple sclerosis patients is a challenging problem, which would benefit from a clinically valid decision support. Stühler et al. presented black box models for this aim which were developed and internally evaluated in a German registry but lacked external validation.
METHODS: In patients from the French OFSEP registry, we independently built and validated models predicting being free of relapse and free of confirmed disability progression (CDP), following the methodological roadmap and predictors reported by Stühler. Hierarchical Bayesian models were fit to predict the outcomes under 6 disease-modifying treatments given the individual disease course up to the moment of treatment change. Data was temporally split on 2017, and models were developed in patients treated earlier (n = 5517). Calibration curves, discrimination, mean squared error (MSE) and relative percentage of root MSE (RMSE%) were assessed by external validation of models in more-recent patients (n = 3768). Non-Bayesian fixed-effects GLMs were also applied and their outcomes were compared to these of the Bayesian ones. For both, we modelled the number of on-therapy relapses with a negative binomial distribution, and CDP occurrence with a binomial distribution.
RESULTS: The performance of our temporally-validated relapse model (MSE: 0.326, C-Index: 0.639) is potentially superior to that of Stühler\'s (MSE: 0.784, C-index: 0.608). Calibration plots revealed miscalibration. Our CDP model (MSE: 0.072, C-Index: 0.777) was also better than its counterpart (MSE: 0.131, C-index: 0.554). Results from non-Bayesian fixed-effects GLM models were similar to the Bayesian ones.
CONCLUSIONS: The relapse and CDP models rebuilt and externally validated in independent data could compare and strengthen the credibility of the Stühler models. Their model-building strategy was replicable.
METHODS: In patients from the French OFSEP registry, we independently built and validated models predicting being free of relapse and free of confirmed disability progression (CDP), following the methodological roadmap and predictors reported by Stühler. Hierarchical Bayesian models were fit to predict the outcomes under 6 disease-modifying treatments given the individual disease course up to the moment of treatment change. Data was temporally split on 2017, and models were developed in patients treated earlier (n = 5517). Calibration curves, discrimination, mean squared error (MSE) and relative percentage of root MSE (RMSE%) were assessed by external validation of models in more-recent patients (n = 3768). Non-Bayesian fixed-effects GLMs were also applied and their outcomes were compared to these of the Bayesian ones. For both, we modelled the number of on-therapy relapses with a negative binomial distribution, and CDP occurrence with a binomial distribution.
RESULTS: The performance of our temporally-validated relapse model (MSE: 0.326, C-Index: 0.639) is potentially superior to that of Stühler\'s (MSE: 0.784, C-index: 0.608). Calibration plots revealed miscalibration. Our CDP model (MSE: 0.072, C-Index: 0.777) was also better than its counterpart (MSE: 0.131, C-index: 0.554). Results from non-Bayesian fixed-effects GLM models were similar to the Bayesian ones.
CONCLUSIONS: The relapse and CDP models rebuilt and externally validated in independent data could compare and strengthen the credibility of the Stühler models. Their model-building strategy was replicable.
摘要：
背景：复发缓解型多发性硬化症患者的个体化治疗和优化治疗是一个具有挑战性的问题，这将受益于临床有效的决策支持。Stühler等人。提出了为此目的的黑盒模型，这些模型是在德国注册表中开发和内部评估的，但缺乏外部验证。
方法：在来自法国OFSEP注册的患者中，我们独立建立并验证了预测无复发和无确认残疾进展（CDP）的模型，遵循Stühler报告的方法路线图和预测因子。分层贝叶斯模型适用于预测6种疾病修饰治疗下的结果，考虑到治疗变化的时刻的各个疾病过程。数据在2017年进行了时间分割，并在早期治疗的患者中开发了模型（n=5517）。校正曲线,歧视,在近期患者(n=3768)中,通过模型的外部验证评估了均方误差(MSE)和根MSE的相对百分比(RMSE%).还应用了非贝叶斯固定效应GLM，并将其结果与贝叶斯结果进行了比较。对于两者来说，我们用负二项分布模拟了治疗中复发的次数，和二项分布的CDP发生。
结果：我们的时间验证的复发模型（MSE：0.326，C指数：0.639）的性能可能优于Stühler的（MSE：0.784，C指数：0.608）。校准图显示校准错误。我们的CDP模型（MSE：0.072，C指数：0.777）也优于其对应模型（MSE：0.131，C指数：0.554）。非贝叶斯固定效应GLM模型的结果与贝叶斯模型相似。
结论：在独立数据中重建和外部验证的复发和CDP模型可以比较并增强Stühler模型的可信度。他们的模型构建策略是可复制的。
方法：在来自法国OFSEP注册的患者中，我们独立建立并验证了预测无复发和无确认残疾进展（CDP）的模型，遵循Stühler报告的方法路线图和预测因子。分层贝叶斯模型适用于预测6种疾病修饰治疗下的结果，考虑到治疗变化的时刻的各个疾病过程。数据在2017年进行了时间分割，并在早期治疗的患者中开发了模型（n=5517）。校正曲线,歧视,在近期患者(n=3768)中,通过模型的外部验证评估了均方误差(MSE)和根MSE的相对百分比(RMSE%).还应用了非贝叶斯固定效应GLM，并将其结果与贝叶斯结果进行了比较。对于两者来说，我们用负二项分布模拟了治疗中复发的次数，和二项分布的CDP发生。
结果：我们的时间验证的复发模型（MSE：0.326，C指数：0.639）的性能可能优于Stühler的（MSE：0.784，C指数：0.608）。校准图显示校准错误。我们的CDP模型（MSE：0.072，C指数：0.777）也优于其对应模型（MSE：0.131，C指数：0.554）。非贝叶斯固定效应GLM模型的结果与贝叶斯模型相似。
结论：在独立数据中重建和外部验证的复发和CDP模型可以比较并增强Stühler模型的可信度。他们的模型构建策略是可复制的。
