PDF(Pubmed)
Abstract：
BACKGROUND: Large language models have propelled recent advances in artificial intelligence technology, facilitating the extraction of medical information from unstructured data such as medical records. Although named entity recognition (NER) is used to extract data from physicians\' records, it has yet to be widely applied to pharmaceutical care records.
OBJECTIVE: In this study, we aimed to investigate the feasibility of automatic extraction of the information regarding patients\' diseases and symptoms from pharmaceutical care records. The verification was performed using Medical Named Entity Recognition-Japanese (MedNER-J), a Japanese disease-extraction system designed for physicians\' records.
METHODS: MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F1-score.
RESULTS: The F1-scores of NER for subjective, objective, assessment, and plan data were 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive-negative classification, the F1-scores were 0.28, 0.39, 0.64, and 0.077, respectively. The F1-scores of NER for objective (0.70) and assessment data (0.76) were higher than those for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F1-score of NER and positive-negative classification was high for assessment data alone (F1-score=0.64), which was attributed to the similarity of its description format and contents to those of the training data.
CONCLUSIONS: MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records.
OBJECTIVE: In this study, we aimed to investigate the feasibility of automatic extraction of the information regarding patients\' diseases and symptoms from pharmaceutical care records. The verification was performed using Medical Named Entity Recognition-Japanese (MedNER-J), a Japanese disease-extraction system designed for physicians\' records.
METHODS: MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F1-score.
RESULTS: The F1-scores of NER for subjective, objective, assessment, and plan data were 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive-negative classification, the F1-scores were 0.28, 0.39, 0.64, and 0.077, respectively. The F1-scores of NER for objective (0.70) and assessment data (0.76) were higher than those for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F1-score of NER and positive-negative classification was high for assessment data alone (F1-score=0.64), which was attributed to the similarity of its description format and contents to those of the training data.
CONCLUSIONS: MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records.
摘要：
背景：大型语言模型推动了人工智能技术的最新进展，便于从医疗记录等非结构化数据中提取医疗信息。虽然命名实体识别(NER)用于从医生记录中提取数据，它尚未广泛应用于药物护理记录。
目的：在本研究中，我们的目的是探讨从药学监护记录中自动提取患者疾病和症状信息的可行性。使用医学命名实体识别-日语（MedNER-J）进行验证，一种为医生记录设计的日本疾病提取系统。
方法：MedNER-J应用于主观，目标,评估，和计划数据来自2018年4月至2019年3月在Keio大学医院接受头孢唑林钠注射液治疗的49例患者的护理记录.MedNER-J的性能在精度方面进行了评估，召回，和F1得分。
结果：NER的F1分数为主观，目标,评估，和计划数据分别为0.46,0.70,0.76和0.35.在NER和正负分类中，F1评分分别为0.28,0.39,0.64和0.077.NER对客观数据（0.70）和评估数据（0.76）的F1得分高于主观数据和计划数据，这支持了NER绩效对客观和评估数据的优越性。这可能是因为客观和评估数据包含许多技术术语，类似于MedNER-J的训练数据。同时,对于单独的评估数据，NER和阳性-阴性分类的F1得分较高（F1得分=0.64），这归因于其描述格式和内容与训练数据的相似性。
结论：MedNER-J成功读取了药学服务记录，并显示了评估数据的最佳表现。然而,在分析评估数据以外的记录方面仍然存在挑战。因此,有必要加强主观数据的培训数据，以便将系统应用于药学服务记录。
目的：在本研究中，我们的目的是探讨从药学监护记录中自动提取患者疾病和症状信息的可行性。使用医学命名实体识别-日语（MedNER-J）进行验证，一种为医生记录设计的日本疾病提取系统。
方法：MedNER-J应用于主观，目标,评估，和计划数据来自2018年4月至2019年3月在Keio大学医院接受头孢唑林钠注射液治疗的49例患者的护理记录.MedNER-J的性能在精度方面进行了评估，召回，和F1得分。
结果：NER的F1分数为主观，目标,评估，和计划数据分别为0.46,0.70,0.76和0.35.在NER和正负分类中，F1评分分别为0.28,0.39,0.64和0.077.NER对客观数据（0.70）和评估数据（0.76）的F1得分高于主观数据和计划数据，这支持了NER绩效对客观和评估数据的优越性。这可能是因为客观和评估数据包含许多技术术语，类似于MedNER-J的训练数据。同时,对于单独的评估数据，NER和阳性-阴性分类的F1得分较高（F1得分=0.64），这归因于其描述格式和内容与训练数据的相似性。
结论：MedNER-J成功读取了药学服务记录，并显示了评估数据的最佳表现。然而,在分析评估数据以外的记录方面仍然存在挑战。因此,有必要加强主观数据的培训数据，以便将系统应用于药学服务记录。
