视网膜疾病是当今主要的公共卫生问题之一,理应需要先进的计算机辅助诊断。我们提出了一种多标签分类的混合模型,由此从光学相干断层扫描(OCT)图像中自动分类七种视网膜疾病。我们证明,通过结合卷积神经网络(CNN)和视觉变压器(VET)的优势,我们可以产生一种更强大的医学图像分类模型,尤其是在考虑视网膜疾病等局部病变信息时。CNN确实被证明在参数利用率方面是有效的,并且通过卷积运算提供提取局部特征和多尺度特征图的能力。另一方面,ViT的自我注意过程允许处理图像中的远程和全局依赖关系。该论文清楚地表明,这些互补能力的杂交(CNNs-VITs)具有更强大和高效的高图像处理潜力。所提出的模型采用称为卷积补丁和令牌嵌入(CPTE)的分层CNN模块,而不是使用变换器中的原始输入OCT图像采用直接令牌化方法。CPTE模块的作用是包含感应偏置,为了减少对大规模数据集的依赖,并解决ViT的低级特征提取挑战。此外,考虑到OCT图像中局部病变信息的重要性,该模型依赖于一个名为ResidualDepthwise-PointwiseConvNet(RDP-ConvNet)的并行模块来提取高级特征。RDP-ConvNet在残差网络体系结构中利用深度和逐点卷积层。HTC-Retina模型的总体性能在三个数据集上进行了评估:OCT-2017、OCT-C8和OCT-2014;优于以前建立的模型,准确率达到99.40%,97.00%,99.77%,分别为99.41%和敏感率,97.00%,99.77%,分别。值得注意的是,该模型表现出高性能,同时保持计算效率。
Retinal diseases are among nowadays major public health issues, deservedly needing advanced computer-aided diagnosis. We propose a hybrid model for multi label classification, whereby seven retinal diseases are automatically classified from Optical Coherence Tomography (OCT) images. We show that, by combining the strengths of Convolutional Neural Networks (CNNs) and Visual Transformers (ViTs), we can produce a more powerful type of model for medical image classification, especially when considering local lesion information such as retinal diseases. CNNs are indeed proved to be efficient at parameter utilization and provide the ability to extract local features and multi-scale feature maps through convolutional operations. On the other hand, ViT\'s self-attention procedure allows processing long-range and global dependencies within an image. The paper clearly shows that the hybridization of these complementary capabilities (CNNs-ViTs) presents a high image processing potential that is more robust and efficient. The proposed model adopts a hierarchical CNN module called Convolutional Patch and Token Embedding (CPTE) instead of employing a direct tokenization approach using the raw input OCT image in the transformer. The CPTE module\'s role is to incorporate an inductive bias, to reduce the reliance on large-scale datasets, and to address the low-level feature extraction challenges of the ViT. In addition, considering the importance of local lesion information in OCT images, the model relies on a parallel module called Residual Depthwise-Pointwise ConvNet (RDP-ConvNet) for extracting high-level features. RDP-ConvNet utilizes depthwise and pointwise convolution layers within a residual network architecture. The overall performance of the HTC-Retina model was evaluated on three datasets: the OCT-2017, OCT-C8, and OCT-2014 ; outperforming previous established models, achieving accuracy rates of 99.40%, 97.00%, and 99.77%, respectively ; and sensitivity rates of 99.41%, 97.00%, and 99.77%, respectively. Notably, the model showed high performance while maintaining computational efficiency.