背景:可预防的患者伤害,尤其是用药错误,代表了医疗保健环境中的重大挑战。在高工作量环境中,分配错误的药物通常与外观相似和声音相似的药物混合有关。用自动单位剂量和药物分配系统代替手动分配以减少药物错误在经历高患者周转或频繁剂量变化的临床设施中并不总是可行的。基于人工智能(AI)的药丸识别工具和智能手机应用程序可能会帮助医护人员在未实施更先进的分配系统的情况下识别药丸。
目的:大多数已发表的关于药丸识别的研究都集中在使用传统编码和深度学习方法进行模型开发的理论方面。使用无代码深度学习(CFDL)作为可访问模型开发的实用替代方法,并在旨在帮助临床环境决策的工具中实施此类模型,在很大程度上仍未探索。在这项研究中,我们试图通过调查CFDL是否是使用自定义数据集开发药丸识别模型的可行方法来解决现有文献中的这一差距,然后在各种部署场景中对模型进行全面评估,在多中心临床环境中。此外,我们旨在突出挑战并提出解决方案,以实现药丸识别模型的最佳性能和现实世界适用性,包括在智能手机应用程序上部署时。
方法:使用MicrosoftAzure自定义视觉平台和大型自定义训练数据集开发了药丸识别模型,该数据集从三家参与医院的前30个分配最多的固体口服剂型(SODF)捕获了26,880张图像。制定了全面的内部和外部测试策略,通过在线API调查模型的性能,并使用在WindowsPC和Android上运行的导出的TensorFlowLite模型脱机,使用量身定制的测试智能手机应用程序。此外,模型校准,对颜色特征的依赖程度和设备依赖性进行了全面评估。使用医院药剂师在三个参与的临床中心捕获的图像评估真实世界的表现。
结果:药丸识别模型在MicrosoftAzure自定义视觉平台中显示出高性能,精度为98.7%,95.1%召回,和98.2%的平均精度(MAP),阈值设置为50%。在利用在线API进行内部测试期间,模型精度达到93.7%,88.96%召回,90.81%的F1评分和87.35%的mAP。在WindowsPC上离线测试TensorFlowLite模型显示性能略有下降,精度为91.16%,83.82%召回,F1得分86.18%,mAP为82.55%。在Android应用程序上离线运行的模型的性能进一步降低到86.50%的精度,75.00%的召回,77.83%的F1得分和69.24%的mAP。在通过在线API进行的外部临床测试中,总体精度为83.10%,召回71.39%,F1评分达到75.76%。
结论:我们的研究表明,使用CFDL方法是开发基于AI的药丸识别系统的可行且具有成本效益的方法。尽管遇到了限制,我们的模型表现很好,特别是通过在线API访问时。CFDL的使用促进了跨学科合作,产生了以人为中心的人工智能模型,增强了现实世界的适用性。我们建议,与其努力建立一个普遍适用的药丸识别系统,模型应根据区域处方集中的药物或特定诊所的需求进行定制,这反过来可以提高在这些位置的实际部署的性能。在注重模型开发的同时,通过培训最终用户如何与基于AI的系统进行正确交互以实现利益最大化,采用以人为本的方法至关重要。未来的研究需要完善药丸识别模型,以实现更广泛的适应性。这包括研究图像预处理和优化技术,以增强手持设备上的离线性能和操作。此外,未来的研究应该探索克服CFDL开发局限性的方法,以增强模型的鲁棒性并减少过拟合。该领域研究人员之间的合作努力以及最佳实践的共享对于改善药丸识别系统至关重要。最终提高患者安全和医疗保健结果。
BACKGROUND: Preventable patient harm, particularly medication errors, represent significant challenges in healthcare settings. Dispensing the wrong medication is often associated with mix-up of lookalike and soundalike drugs in high workload environments. Replacing manual dispensing with automated unit dose and medication dispensing systems to reduce medication errors is not always feasible in clinical facilities experiencing high patient turn-around or frequent dose changes. Artificial intelligence (AI) based pill recognition tools and smartphone applications could potentially aid healthcare workers in identifying pills in situations where more advanced dispensing systems are not implemented.
OBJECTIVE: Most of the published research on pill recognition focuses on theoretical aspects of model development using traditional coding and deep learning methods. The use of code-free deep learning (CFDL) as a practical alternative for accessible model development, and implementation of such models in tools intended to aid decision making in clinical settings, remains largely unexplored. In this
study, we sought to address this gap in existing literature by investigating whether CFDL is a viable approach for developing pill recognition models using a custom dataset, followed by a thorough evaluation of the model across various deployment scenarios, and in multicenter clinical settings. Furthermore, we aimed to highlight challenges and propose solutions to achieve optimal performance and real-world applicability of pill recognition models, including when deployed on smartphone applications.
METHODS: A pill recognition model was developed utilizing Microsoft Azure Custom Vision platform and a large custom training dataset of 26,880 images captured from the top 30 most dispensed solid oral dosage forms (SODFs) at the three participating hospitals. A comprehensive internal and external testing strategy was devised, model\'s performance was investigated through the online API, and offline using exported TensorFlow Lite model running on a Windows PC and on Android, using a tailor-made testing smartphone application. Additionally, model\'s calibration, degree of reliance on color features and device dependency was thoroughly evaluated. Real-world performance was assessed using images captured by hospital pharmacists at three participating clinical centers.
RESULTS: The pill recognition model showed high performance in Microsoft Azure Custom Vision platform with 98.7 % precision, 95.1 % recall, and 98.2 % mean average precision (mAP), with thresholds set to 50 %. During internal testing utilizing the online API, the model reached 93.7 % precision, 88.96 % recall, 90.81 % F1-score and 87.35 % mAP. Testing the offline TensorFlow Lite model on Windows PC showed a slight performance reduction, with 91.16 % precision, 83.82 % recall, 86.18 % F1-score and 82.55 % mAP. Performance of the model running offline on the Android application was further reduced to 86.50 % precision, 75.00 % recall, 77.83 % F1-score and 69.24 % mAP. During external clinical testing through the online API an overall precision of 83.10 %, recall of 71.39 %, and F1-score of 75.76 % was achieved.
CONCLUSIONS: Our
study demonstrates that using a CFDL approach is a feasible and cost-effective method for developing AI-based pill recognition systems. Despite the limitations encountered, our model performed well, particularly when accessed through the online API. The use of CFDL facilitates interdisciplinary collaboration, resulting in human-centered AI models with enhanced real-world applicability. We suggest that rather than striving to build a universally applicable pill recognition system, models should be tailored to the medications in a regional formulary or needs of a specific clinic, which can in turn lead to improved performance in real-world deployment in these locations. Parallel to focusing on model development, it is crucial to employ a human centered approach by training the end users on how to properly interact with the AI based system to maximize benefits. Future research is needed on refining pill recognition models for broader adaptability. This includes investigating image pre-processing and optimization techniques to enhance offline performance and operation on handheld devices. Moreover, future studies should explore methods to overcome limitations of CFDL development to enhance the robustness of models and reduce overfitting. Collaborative efforts between researchers in this domain and sharing of best practices are vital to improve pill recognition systems, ultimately enhancing patient safety and healthcare outcomes.