目的:我们以前已经开发了分级指标来客观地衡量内镜医师在内镜套管胃成形术(ESG)中的表现。我们的主要目标之一是自动化衡量绩效的过程。为了实现这一目标,正在执行的重复任务(抓握或缝合)以及内窥镜缝合装置在胃中的位置(Incisura,前壁,更大的曲率,或后壁)需要准确记录。
方法:对于本研究,我们使用来自对离体猪标本进行ESG程序的专家的屏幕截图和视频剪辑来填充我们的数据集。数据增强被用来扩大我们的数据集,和合成少数过采样(SMOTE)来平衡它。我们使用图像的深度学习和视频的计算机视觉对胃的部分进行了胃定位和任务分类。
结果:在没有SMOTE图像的情况下从内窥镜对胃的位置进行分类,测试和验证准确率分别为89%和84%,分别。为了用SMOTE从内窥镜对胃的位置进行分类,图像的准确率分别为97%和90%,而对于视频,测试和验证的准确度分别为99%和98%,分别。对于任务分类,图像的准确率分别为97%和89%,而对于视频,测试和验证的准确度都是100%,分别。
结论:我们以97%的训练准确率对ESG过程中操作的四个不同的胃部分进行了分类,并利用图像以99%的训练准确率对两个重复任务进行了分类。我们还以99%的训练精度对胃的四个部分进行了分类,并以视频帧的100%的训练精度对两个重复任务进行了分类。这项工作对于ESG学习者的自动化反馈机制至关重要。
OBJECTIVE: We have previously developed grading metrics to objectively measure endoscopist performance in endoscopic sleeve gastroplasty (ESG). One of our primary goals is to automate the process of measuring performance. To achieve this goal, the repeated task being performed (grasping or suturing) and the location of the endoscopic suturing device in the stomach (Incisura, Anterior Wall, Greater Curvature, or Posterior Wall) need to be accurately recorded.
METHODS: For this study, we populated our dataset using screenshots and video clips from experts carrying out the ESG procedure on ex vivo porcine specimens. Data augmentation was used to enlarge our dataset, and synthetic minority oversampling (SMOTE) to balance it. We performed stomach localization for parts of the stomach and task classification using deep learning for images and computer vision for videos.
RESULTS: Classifying the stomach\'s location from the endoscope without SMOTE for images resulted in 89% and 84% testing and validation accuracy, respectively. For classifying the location of the stomach from the endoscope with SMOTE, the accuracies were 97% and 90% for images, while for videos, the accuracies were 99% and 98% for testing and validation, respectively. For task classification, the accuracies were 97% and 89% for images, while for videos, the accuracies were 100% for both testing and validation, respectively.
CONCLUSIONS: We classified the four different stomach parts manipulated during the ESG procedure with 97% training accuracy and classified two repeated tasks with 99% training accuracy with images. We also classified the four parts of the stomach with a 99% training accuracy and two repeated tasks with a 100% training accuracy with video frames. This work will be essential in automating feedback mechanisms for learners in ESG.