医学图像分析在临床诊断中起着重要的作用。在本文中,我们研究了最近在医学图像上的分段任意模型(SAM),并报告九种医学图像分割基准的定量和定性零镜头分割结果,涵盖各种成像模式,如光学相干断层扫描(OCT),磁共振成像(MRI),和计算机断层扫描(CT),以及不同的应用,包括皮肤病学,眼科,和放射学。这些基准具有代表性,通常用于模型开发。我们的实验结果表明,虽然SAM在一般领域的图像上表现出显著的分割性能,对于分布外的图像,其零镜头分割能力仍然受到限制,例如,医学图像。此外,SAM在不同的未见过的医学领域中表现出不一致的零镜头分割性能。对于某些结构化目标,例如,血管,SAM的零射分割完全失败。相比之下,用少量数据对其进行简单的微调可以显着提高分割质量,显示了使用微调SAM实现精确医学图像分割以进行精确诊断的巨大潜力和可行性。我们的研究表明,通用视觉基础模型在医学成像上的多功能性,和他们的巨大潜力,以实现所需的性能,通过微调,并最终解决与访问大型和多样化的医疗数据集以支持临床诊断相关的挑战。
Medical image analysis plays an important role in clinical diagnosis. In this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.