PDF(Pubmed)
Abstract：
BACKGROUND: This study aims to propose a semi-automatic method for monitoring the waiting times of follow-up examinations within the National Health System (NHS) in Italy, which is currently not possible to due the absence of the necessary structured information in the official databases.
METHODS: A Natural Language Processing (NLP) based pipeline has been developed to extract the waiting time information from the text of referrals for follow-up examinations in the Lombardy Region. A manually annotated dataset of 10 000 referrals has been used to develop the pipeline and another manually annotated dataset of 10 000 referrals has been used to test its performance. Subsequently, the pipeline has been used to analyze all 12 million referrals prescribed in 2021 and performed by May 2022 in the Lombardy Region.
RESULTS: The NLP-based pipeline exhibited high precision (0.999) and recall (0.973) in identifying waiting time information from referrals\' texts, with high accuracy in normalization (0.948-0.998). The overall reporting of timing indications in referrals\' texts for follow-up examinations was low (2%), showing notable variations across medical disciplines and types of prescribing physicians. Among the referrals reporting waiting times, 16% experienced delays (average delay = 19 days, standard deviation = 34 days), with significant differences observed across medical disciplines and geographical areas.
CONCLUSIONS: The use of NLP proved to be a valuable tool for assessing waiting times in follow-up examinations, which are particularly critical for the NHS due to the significant impact of chronic diseases, where follow-up exams are pivotal. Health authorities can exploit this tool to monitor the quality of NHS services and optimize resource allocation.
METHODS: A Natural Language Processing (NLP) based pipeline has been developed to extract the waiting time information from the text of referrals for follow-up examinations in the Lombardy Region. A manually annotated dataset of 10 000 referrals has been used to develop the pipeline and another manually annotated dataset of 10 000 referrals has been used to test its performance. Subsequently, the pipeline has been used to analyze all 12 million referrals prescribed in 2021 and performed by May 2022 in the Lombardy Region.
RESULTS: The NLP-based pipeline exhibited high precision (0.999) and recall (0.973) in identifying waiting time information from referrals\' texts, with high accuracy in normalization (0.948-0.998). The overall reporting of timing indications in referrals\' texts for follow-up examinations was low (2%), showing notable variations across medical disciplines and types of prescribing physicians. Among the referrals reporting waiting times, 16% experienced delays (average delay = 19 days, standard deviation = 34 days), with significant differences observed across medical disciplines and geographical areas.
CONCLUSIONS: The use of NLP proved to be a valuable tool for assessing waiting times in follow-up examinations, which are particularly critical for the NHS due to the significant impact of chronic diseases, where follow-up exams are pivotal. Health authorities can exploit this tool to monitor the quality of NHS services and optimize resource allocation.
摘要：
背景：本研究旨在提出一种半自动方法，用于在意大利国家卫生系统（NHS）内监测随访检查的等待时间，由于官方数据库中缺乏必要的结构化信息，目前尚不可能。
方法：已经开发了一种基于自然语言处理（NLP）的管道，用于从推荐文本中提取等待时间信息，以便在伦巴第地区进行后续检查。10.000个推荐的手动注释数据集已用于开发管道，而10.000个推荐的另一个手动注释数据集已用于测试其性能。随后，该管道已用于分析2021年规定的所有1200万次推荐，并于2022年5月在伦巴第大区进行。
结果：基于NLP的管道在从推荐文本中识别等待时间信息方面表现出高精度（0.999）和召回率（0.973），归一化精度高(0.948-0.998)。随访检查转介文本中时间指示的总体报告较低(2%)，显示出不同医学学科和处方医生类型的显着差异。在报告等待时间的推荐中，16%的人经历了延误(平均延误=19天，标准偏差=34天)，在医学学科和地理区域之间观察到显著差异。
结论：使用NLP被证明是评估后续检查等待时间的宝贵工具，由于慢性病的重大影响，这对NHS尤其重要，后续考试至关重要。卫生当局可以利用此工具来监控NHS服务的质量并优化资源分配。
方法：已经开发了一种基于自然语言处理（NLP）的管道，用于从推荐文本中提取等待时间信息，以便在伦巴第地区进行后续检查。10.000个推荐的手动注释数据集已用于开发管道，而10.000个推荐的另一个手动注释数据集已用于测试其性能。随后，该管道已用于分析2021年规定的所有1200万次推荐，并于2022年5月在伦巴第大区进行。
结果：基于NLP的管道在从推荐文本中识别等待时间信息方面表现出高精度（0.999）和召回率（0.973），归一化精度高(0.948-0.998)。随访检查转介文本中时间指示的总体报告较低(2%)，显示出不同医学学科和处方医生类型的显着差异。在报告等待时间的推荐中，16%的人经历了延误(平均延误=19天，标准偏差=34天)，在医学学科和地理区域之间观察到显著差异。
结论：使用NLP被证明是评估后续检查等待时间的宝贵工具，由于慢性病的重大影响，这对NHS尤其重要，后续考试至关重要。卫生当局可以利用此工具来监控NHS服务的质量并优化资源分配。
