强化学习(RL)最近在医疗保健领域发现了许多应用,这要归功于它对临床决策的自然适应以及从观察数据中学习最佳决策的能力。在临床实践中采用基于RL的解决方案的关键挑战,然而,是将现有知识纳入学习合适的解决方案。现有的知识,例如医疗指南可以提高解决方案的安全性,在患者的短期和长期结果之间取得更好的平衡,并增加临床医生的信任和采纳。我们提出了一个框架,用于在RL中纳入医学指南中提供的知识。该框架包括用于执行安全约束的组件,以及根据这些指南改变学习信号以更好地平衡短期和长期结果的方法。我们通过使用临床建立的通气指南扩展现有的基于RL的机械通气(MV)方法来评估框架。政策外政策评估的结果表明,我们的方法有可能降低90天死亡率,同时确保肺保护性通气。该框架为在临床实践中实施RL提供了重要的垫脚石,并为进一步研究开辟了多种途径。
Reinforcement Learning (RL) has recently found many applications in the healthcare domain thanks to its natural fit to clinical decision-making and ability to learn optimal decisions from observational data. A key challenge in adopting RL-based solution in clinical practice, however, is the inclusion of existing knowledge in learning a suitable solution. Existing knowledge from e.g. medical
guidelines may improve the safety of solutions, produce a better balance between short- and long-term outcomes for patients and increase trust and adoption by clinicians. We present a framework for including knowledge available from medical
guidelines in RL. The framework includes components for enforcing safety constraints and an approach that alters the learning signal to better balance short- and long-term outcomes based on these
guidelines. We evaluate the framework by extending an existing RL-based mechanical ventilation (MV) approach with clinically established ventilation
guidelines. Results from off-policy policy evaluation indicate that our approach has the potential to decrease 90-day mortality while ensuring lung protective ventilation. This framework provides an important stepping stone towards implementations of RL in clinical practice and opens up several avenues for further research.