目的:内窥镜图像中的镜面反射不仅会干扰视觉感知,还会妨碍计算机视觉算法的性能。然而,这些反射的复杂性质和可变性,再加上缺乏相关数据集,对移除构成持续的挑战。
方法:我们介绍EndoSRR,一种用于消除内窥镜图像中的镜面反射的鲁棒方法。EndoSRR包括两个阶段:反射检测和反射区域填充。在反射检测阶段,我们使用弱标记的数据集调整和微调分段任何模型(SAM),实现一个准确的反射掩模。对于反射区域,我们雇佣了LaMa,在4.5M图像数据集上训练的基于快速傅立叶卷积的模型,能够对任意形状的反射区域进行有效的绘画。最后,我们引入了双预先训练模型的迭代优化策略,以完善镜面反射去除的结果,名为DPMIO。
结果:利用SCARED-2019数据集,我们的方法在定性和定量评估方面都超过了最先进的方法.定性,我们的方法擅长精确检测反射区域,产生更自然和现实的绘画结果。定量地,我们的方法在两个分割评估指标(IoU,电子测量,等。)和图像修复评估指标(PSNR,SSIM,等。).
结论:实验结果强调了熟练的内窥镜镜面反射去除对增强视觉感知和下游任务的重要性。本研究中提出的方法和结果有望促进镜面反射去除的进步,从而提高微创手术的准确性和安全性。
OBJECTIVE: Specular reflections in endoscopic images not only disturb visual perception but also hamper computer vision algorithm performance. However, the intricate nature and variability of these reflections, coupled with a lack of relevant datasets, pose ongoing challenges for removal.
METHODS: We present EndoSRR, a robust method for eliminating specular reflections in endoscopic images. EndoSRR comprises two stages: reflection detection and reflection region inpainting. In the reflection detection stage, we adapt and fine-tune the segment anything model (SAM) using a weakly labeled dataset, achieving an accurate reflection mask. For reflective region inpainting, we employ LaMa, a fast Fourier convolution-based model trained on a 4.5M-image dataset, enabling effective inpainting of arbitrarily shaped reflection regions. Lastly, we introduce an iterative optimization strategy for dual pre-trained models to refine the results of specular reflection removal, named DPMIO.
RESULTS: Utilizing the SCARED-2019 dataset, our approach surpasses state-of-the-art methods in both qualitative and quantitative evaluations. Qualitatively, our method excels in accurately detecting reflective regions, yielding more natural and realistic inpainting results. Quantitatively, our method demonstrates superior performance in both segmentation evaluation metrics (IoU, E-measure, etc.) and image inpainting evaluation metrics (PSNR, SSIM, etc.).
CONCLUSIONS: The experimental results underscore the significance of proficient endoscopic specular reflection removal for enhancing visual perception and downstream tasks. The methodology and results presented in this study are poised to catalyze advancements in specular reflection removal, thereby augmenting the accuracy and safety of minimally invasive surgery.