PDF(Pubmed)
Abstract：
BACKGROUND: For time-to-event endpoints, three additional benefit assessment methods have been developed aiming at an unbiased knowledge about the magnitude of clinical benefit of newly approved treatments. The American Society of Clinical Oncology (ASCO) defines a continuous score using the hazard ratio point estimate (HR-PE). The European Society for Medical Oncology (ESMO) and the German Institute for Quality and Efficiency in Health Care (IQWiG) developed methods with an ordinal outcome using lower and upper limits of the 95% HR confidence interval (HR-CI), respectively. We describe all three frameworks for additional benefit assessment aiming at a fair comparison across different stakeholders. Furthermore, we determine which ASCO score is consistent with which ESMO/IQWiG category.
METHODS: In a comprehensive simulation study with different failure time distributions and treatment effects, we compare all methods using Spearman\'s correlation and descriptive measures. For determination of ASCO values consistent with categories of ESMO/IQWiG, maximizing weighted Cohen\'s Kappa approach was used.
RESULTS: Our research depicts a high positive relationship between ASCO/IQWiG and a low positive relationship between ASCO/ESMO. An ASCO score smaller than 17, 17 to 20, 20 to 24, and greater than 24 corresponds to ESMO categories. Using ASCO values of 21 and 38 as cutoffs represents IQWiG categories.
CONCLUSIONS: We investigated the statistical aspects of the methods and hence implemented slightly reduced versions of all methods.
CONCLUSIONS: IQWiG and ASCO are more conservative than ESMO, which often awards the maximal category independent of the true effect and is at risk of overcompensating with various failure time distributions. ASCO has similar characteristics as IQWiG. Delayed treatment effects and underpowered/overpowered studies influence all methods in some degree. Nevertheless, ESMO is the most liberal one.
CONCLUSIONS: For the additional benefit assessment, the American Society of Clinical Oncology (ASCO) uses the hazard ratio point estimate (HR-PE) for their continuous score. In contrast, the European Society for Medical Oncology (ESMO) and the German Institute for Quality and Efficiency in Health Care (IQWiG) use the lower and upper 95% HR confidence interval (HR-CI) to specific thresholds, respectively. ESMO generously assigns maximal scores, while IQWiG is more conservative.This research provides the first comparison between IQWiG and ASCO and describes all three frameworks for additional benefit assessment aiming for a fair comparison across different stakeholders. Furthermore, thresholds for ASCO consistent with ESMO and IQWiG categories are determined, enabling a comparison of the methods in practice in a fair manner.IQWiG and ASCO are the more conservative methods, while ESMO awards high percentages of maximal categories, especially with various failure time distributions. ASCO has similar characteristics as IQWiG. Delayed treatment effects and under/-overpowered studies influence all methods. Nevertheless, ESMO is the most liberal one. An ASCO score smaller than 17, 17 to 20, 20 to 24, and greater than 24 correspond to the categories of ESMO. Using ASCO values of 21 and 38 as cutoffs represents categories of IQWiG.
METHODS: In a comprehensive simulation study with different failure time distributions and treatment effects, we compare all methods using Spearman\'s correlation and descriptive measures. For determination of ASCO values consistent with categories of ESMO/IQWiG, maximizing weighted Cohen\'s Kappa approach was used.
RESULTS: Our research depicts a high positive relationship between ASCO/IQWiG and a low positive relationship between ASCO/ESMO. An ASCO score smaller than 17, 17 to 20, 20 to 24, and greater than 24 corresponds to ESMO categories. Using ASCO values of 21 and 38 as cutoffs represents IQWiG categories.
CONCLUSIONS: We investigated the statistical aspects of the methods and hence implemented slightly reduced versions of all methods.
CONCLUSIONS: IQWiG and ASCO are more conservative than ESMO, which often awards the maximal category independent of the true effect and is at risk of overcompensating with various failure time distributions. ASCO has similar characteristics as IQWiG. Delayed treatment effects and underpowered/overpowered studies influence all methods in some degree. Nevertheless, ESMO is the most liberal one.
CONCLUSIONS: For the additional benefit assessment, the American Society of Clinical Oncology (ASCO) uses the hazard ratio point estimate (HR-PE) for their continuous score. In contrast, the European Society for Medical Oncology (ESMO) and the German Institute for Quality and Efficiency in Health Care (IQWiG) use the lower and upper 95% HR confidence interval (HR-CI) to specific thresholds, respectively. ESMO generously assigns maximal scores, while IQWiG is more conservative.This research provides the first comparison between IQWiG and ASCO and describes all three frameworks for additional benefit assessment aiming for a fair comparison across different stakeholders. Furthermore, thresholds for ASCO consistent with ESMO and IQWiG categories are determined, enabling a comparison of the methods in practice in a fair manner.IQWiG and ASCO are the more conservative methods, while ESMO awards high percentages of maximal categories, especially with various failure time distributions. ASCO has similar characteristics as IQWiG. Delayed treatment effects and under/-overpowered studies influence all methods. Nevertheless, ESMO is the most liberal one. An ASCO score smaller than 17, 17 to 20, 20 to 24, and greater than 24 correspond to the categories of ESMO. Using ASCO values of 21 and 38 as cutoffs represents categories of IQWiG.
摘要：
背景：对于时间到事件的端点，另外还开发了3种获益评估方法,目的是对新批准的治疗方法的临床获益程度进行无偏见了解.美国临床肿瘤学会（ASCO）使用风险比点估计（HR-PE）定义了连续评分。欧洲医学肿瘤学会（ESMO）和德国医疗保健质量和效率研究所（IQWiG）开发了使用95％HR置信区间（HR-CI）的下限和上限的顺序结果的方法。分别。我们描述了用于额外利益评估的所有三个框架，旨在对不同利益相关者进行公平比较。此外，我们确定哪个ASCO评分与哪个ESMO/IQWiG类别一致.
方法：在具有不同故障时间分布和处理效果的综合模拟研究中，我们使用Spearman相关性和描述性度量比较了所有方法。为了确定符合ESMO/IQWiG类别的ASCO值，使用了最大化加权科恩的Kappa方法。
结果：我们的研究描述了ASCO/IQWiG之间的高度正相关和ASCO/ESMO之间的低正相关。小于17、17至20、20至24和大于24的ASCO分数对应于ESMO类别。使用ASCO值21和38作为截止值表示IQWiG类别。
结论：我们调查了方法的统计学方面，因此实施了所有方法的略微缩减版本。
结论：IQWiG和ASCO比ESMO更保守，通常授予与真实效果无关的最大类别，并且有可能因各种故障时间分布而过度补偿。ASCO具有与IQWiG相似的特性。延迟治疗效果和动力不足/过度研究在一定程度上影响所有方法。然而，ESMO是最自由的。
结论：对于额外的收益评估，美国临床肿瘤学会(ASCO)使用风险比点估计值(HR-PE)作为其连续评分.相比之下,欧洲医学肿瘤学会（ESMO）和德国医疗保健质量和效率研究所（IQWiG）使用下限和上限95％的HR置信区间（HR-CI）到特定阈值，分别。ESMO慷慨地分配最大分数，而IQWiG更保守。本研究提供了IQWiG和ASCO之间的首次比较，并描述了所有三个额外收益评估框架，旨在对不同利益相关者进行公平比较。此外，确定符合ESMO和IQWiG类别的ASCO阈值，能够以公平的方式对实践中的方法进行比较。IQWiG和ASCO是比较保守的方法，虽然ESMO授予最高类别的高百分比，特别是在各种故障时间分布的情况下。ASCO具有与IQWiG相似的特性。延迟的治疗效果和/-过度研究影响所有方法。然而，ESMO是最自由的。小于17、17至20、20至24和大于24的ASCO分数对应于ESMO的类别。使用ASCO值21和38作为截止值表示IQWiG的类别。
方法：在具有不同故障时间分布和处理效果的综合模拟研究中，我们使用Spearman相关性和描述性度量比较了所有方法。为了确定符合ESMO/IQWiG类别的ASCO值，使用了最大化加权科恩的Kappa方法。
结果：我们的研究描述了ASCO/IQWiG之间的高度正相关和ASCO/ESMO之间的低正相关。小于17、17至20、20至24和大于24的ASCO分数对应于ESMO类别。使用ASCO值21和38作为截止值表示IQWiG类别。
结论：我们调查了方法的统计学方面，因此实施了所有方法的略微缩减版本。
结论：IQWiG和ASCO比ESMO更保守，通常授予与真实效果无关的最大类别，并且有可能因各种故障时间分布而过度补偿。ASCO具有与IQWiG相似的特性。延迟治疗效果和动力不足/过度研究在一定程度上影响所有方法。然而，ESMO是最自由的。
结论：对于额外的收益评估，美国临床肿瘤学会(ASCO)使用风险比点估计值(HR-PE)作为其连续评分.相比之下,欧洲医学肿瘤学会（ESMO）和德国医疗保健质量和效率研究所（IQWiG）使用下限和上限95％的HR置信区间（HR-CI）到特定阈值，分别。ESMO慷慨地分配最大分数，而IQWiG更保守。本研究提供了IQWiG和ASCO之间的首次比较，并描述了所有三个额外收益评估框架，旨在对不同利益相关者进行公平比较。此外，确定符合ESMO和IQWiG类别的ASCO阈值，能够以公平的方式对实践中的方法进行比较。IQWiG和ASCO是比较保守的方法，虽然ESMO授予最高类别的高百分比，特别是在各种故障时间分布的情况下。ASCO具有与IQWiG相似的特性。延迟的治疗效果和/-过度研究影响所有方法。然而，ESMO是最自由的。小于17、17至20、20至24和大于24的ASCO分数对应于ESMO的类别。使用ASCO值21和38作为截止值表示IQWiG的类别。
