PDF(Pubmed)
Abstract：
UNASSIGNED: Hyperspectral imaging (HSI) is an emerging imaging modality for oncological applications and can improve cancer detection with digital pathology.
UNASSIGNED: The study aims to highlight the increased accuracy and sensitivity of detecting the margin of thyroid carcinoma in hematoxylin and eosin (H&E)-stained histological slides using HSI and data augmentation methods.
UNASSIGNED: Using an automated microscopic imaging system, we captured 2599 hyperspectral images from 65 H&E-stained human thyroid slides. Images were then preprocessed into 153,906 image patches of dimension 250 × 250 × 84 pixels . We modified the TimeSformer network architecture, which used alternating spectral attention and spatial attention layers. We implemented several data augmentation methods for HSI based on the RandAugment algorithm. We compared the performances of TimeSformer on HSI against the performances of pretrained ConvNext and pretrained vision transformers (ViT) networks on red, green, and blue (RGB) images. Finally, we applied attention unrolling techniques on the trained TimeSformer network to identify the biological features to which the network paid attention.
UNASSIGNED: In the testing dataset, TimeSformer achieved an accuracy of 90.87%, a weighted F 1 score of 89.79%, a sensitivity of 91.50%, and an area under the receiving operator characteristic curve (AU-ROC) score of 97.04%. Additionally, TimeSformer produced thyroid carcinoma tumor margins with an average Jaccard score of 0.76 mm. Without data augmentation, TimeSformer achieved an accuracy of 88.23%, a weighted F 1 score of 86.46%, a sensitivity of 85.53%, and an AU-ROC score of 94.94%. In comparison, the ViT network achieved an 89.98% accuracy, an 88.14% weighted F 1 score, an 84.77% sensitivity, and a 96.17% AU-ROC. Our visualization results showed that the network paid attention to biological features.
UNASSIGNED: The TimeSformer model trained with hyperspectral histological data consistently outperformed conventional RGB-based models, highlighting the superiority of HSI in this context. Our proposed augmentation methods improved the accuracy, the F 1 score, and the sensitivity score.
UNASSIGNED: The study aims to highlight the increased accuracy and sensitivity of detecting the margin of thyroid carcinoma in hematoxylin and eosin (H&E)-stained histological slides using HSI and data augmentation methods.
UNASSIGNED: Using an automated microscopic imaging system, we captured 2599 hyperspectral images from 65 H&E-stained human thyroid slides. Images were then preprocessed into 153,906 image patches of dimension 250 × 250 × 84 pixels . We modified the TimeSformer network architecture, which used alternating spectral attention and spatial attention layers. We implemented several data augmentation methods for HSI based on the RandAugment algorithm. We compared the performances of TimeSformer on HSI against the performances of pretrained ConvNext and pretrained vision transformers (ViT) networks on red, green, and blue (RGB) images. Finally, we applied attention unrolling techniques on the trained TimeSformer network to identify the biological features to which the network paid attention.
UNASSIGNED: In the testing dataset, TimeSformer achieved an accuracy of 90.87%, a weighted F 1 score of 89.79%, a sensitivity of 91.50%, and an area under the receiving operator characteristic curve (AU-ROC) score of 97.04%. Additionally, TimeSformer produced thyroid carcinoma tumor margins with an average Jaccard score of 0.76 mm. Without data augmentation, TimeSformer achieved an accuracy of 88.23%, a weighted F 1 score of 86.46%, a sensitivity of 85.53%, and an AU-ROC score of 94.94%. In comparison, the ViT network achieved an 89.98% accuracy, an 88.14% weighted F 1 score, an 84.77% sensitivity, and a 96.17% AU-ROC. Our visualization results showed that the network paid attention to biological features.
UNASSIGNED: The TimeSformer model trained with hyperspectral histological data consistently outperformed conventional RGB-based models, highlighting the superiority of HSI in this context. Our proposed augmentation methods improved the accuracy, the F 1 score, and the sensitivity score.
摘要：
■高光谱成像（HSI）是用于肿瘤应用的新兴成像模式，可以通过数字病理学改善癌症检测。
■该研究旨在强调使用HSI和数据增强方法在苏木精和伊红（H＆E）染色的组织学切片中检测甲状腺癌边缘的准确性和敏感性。
■使用自动显微成像系统，我们从65个H&E染色的人甲状腺载玻片上捕获了2599个高光谱图像。然后将图像预处理为153,906个尺寸为250×250×84像素的图像块。我们修改了TimeSformer网络架构，使用交替的光谱注意层和空间注意层。我们基于RandAugment算法为HSI实现了几种数据增强方法。我们比较了TimeSformer在HSI上的表现与预训练的ConvNext和预训练的视觉变压器（ViT）网络在红色上的表现，绿色,和蓝色（RGB）图像。最后,我们在经过训练的TimeSformer网络上应用了注意力展开技术来识别网络关注的生物学特征。
■在测试数据集中，TimeSformer实现了90.87%的准确度，加权F1得分为89.79%，灵敏度为91.50%，接受手术者特征曲线下面积(AU-ROC)评分为97.04%。此外,TimeSformer产生甲状腺癌肿瘤边缘，平均Jaccard评分为0.76mm。没有数据增强，TimeSformer的准确率为88.23%，加权F1得分为86.46%，灵敏度为85.53%，AU-ROC得分为94.94%。相比之下,ViT网络达到了89.98%的准确率，加权F1得分88.14%，灵敏度为84.77%，和96.17%的AU-ROC。我们的可视化结果表明，该网络关注生物学特征。
■使用高光谱组织学数据训练的TimeSformer模型始终优于传统的基于RGB的模型，突出了恒生指数在这一背景下的优越性。我们提出的增强方法提高了准确性，F1分,和敏感度得分。
■该研究旨在强调使用HSI和数据增强方法在苏木精和伊红（H＆E）染色的组织学切片中检测甲状腺癌边缘的准确性和敏感性。
■使用自动显微成像系统，我们从65个H&E染色的人甲状腺载玻片上捕获了2599个高光谱图像。然后将图像预处理为153,906个尺寸为250×250×84像素的图像块。我们修改了TimeSformer网络架构，使用交替的光谱注意层和空间注意层。我们基于RandAugment算法为HSI实现了几种数据增强方法。我们比较了TimeSformer在HSI上的表现与预训练的ConvNext和预训练的视觉变压器（ViT）网络在红色上的表现，绿色,和蓝色（RGB）图像。最后,我们在经过训练的TimeSformer网络上应用了注意力展开技术来识别网络关注的生物学特征。
■在测试数据集中，TimeSformer实现了90.87%的准确度，加权F1得分为89.79%，灵敏度为91.50%，接受手术者特征曲线下面积(AU-ROC)评分为97.04%。此外,TimeSformer产生甲状腺癌肿瘤边缘，平均Jaccard评分为0.76mm。没有数据增强，TimeSformer的准确率为88.23%，加权F1得分为86.46%，灵敏度为85.53%，AU-ROC得分为94.94%。相比之下,ViT网络达到了89.98%的准确率，加权F1得分88.14%，灵敏度为84.77%，和96.17%的AU-ROC。我们的可视化结果表明，该网络关注生物学特征。
■使用高光谱组织学数据训练的TimeSformer模型始终优于传统的基于RGB的模型，突出了恒生指数在这一背景下的优越性。我们提出的增强方法提高了准确性，F1分,和敏感度得分。
