PDF(Pubmed)
Abstract：
UNASSIGNED: Single-chip imaging devices featuring vertically stacked photodiodes and pixelated spectral filters are advancing multi-dye imaging methods for cancer surgeries, though this innovation comes with a compromise in spatial resolution. To mitigate this drawback, we developed a deep convolutional neural network (CNN) aimed at demosaicing the color and near-infrared (NIR) channels, with its performance validated on both pre-clinical and clinical datasets.
UNASSIGNED: We introduce an optimized deep CNN designed for demosaicing both color and NIR images obtained using a hexachromatic imaging sensor.
UNASSIGNED: A residual CNN was fine-tuned and trained on a dataset of color images and subsequently assessed on a series of dual-channel, color, and NIR images to demonstrate its enhanced performance compared with traditional bilinear interpolation.
UNASSIGNED: Our optimized CNN for demosaicing color and NIR images achieves a reduction in the mean square error by 37% for color and 40% for NIR, respectively, and enhances the structural dissimilarity index by 37% across both imaging modalities in pre-clinical data. In clinical datasets, the network improves the mean square error by 35% in color images and 42% in NIR images while enhancing the structural dissimilarity index by 39% in both imaging modalities.
UNASSIGNED: We showcase enhancements in image resolution for both color and NIR modalities through the use of an optimized CNN tailored for a hexachromatic image sensor. With the ongoing advancements in graphics card computational power, our approach delivers significant improvements in resolution that are feasible for real-time execution in surgical environments.
UNASSIGNED: We introduce an optimized deep CNN designed for demosaicing both color and NIR images obtained using a hexachromatic imaging sensor.
UNASSIGNED: A residual CNN was fine-tuned and trained on a dataset of color images and subsequently assessed on a series of dual-channel, color, and NIR images to demonstrate its enhanced performance compared with traditional bilinear interpolation.
UNASSIGNED: Our optimized CNN for demosaicing color and NIR images achieves a reduction in the mean square error by 37% for color and 40% for NIR, respectively, and enhances the structural dissimilarity index by 37% across both imaging modalities in pre-clinical data. In clinical datasets, the network improves the mean square error by 35% in color images and 42% in NIR images while enhancing the structural dissimilarity index by 39% in both imaging modalities.
UNASSIGNED: We showcase enhancements in image resolution for both color and NIR modalities through the use of an optimized CNN tailored for a hexachromatic image sensor. With the ongoing advancements in graphics card computational power, our approach delivers significant improvements in resolution that are feasible for real-time execution in surgical environments.
摘要：
■具有垂直堆叠的光电二极管和像素化光谱滤光片的单芯片成像设备正在推进癌症手术的多染料成像方法，尽管这种创新伴随着空间分辨率的妥协。为了减轻这个缺点，我们开发了一个深度卷积神经网络（CNN），旨在对颜色和近红外（NIR）通道进行去马赛克，在临床前和临床数据集上验证了其性能。
■我们引入了一种优化的深度CNN，旨在对使用六色成像传感器获得的彩色和近红外图像进行去马赛克。
■在彩色图像数据集上对残差CNN进行了微调和训练，随后在一系列双通道上进行了评估，颜色,和近红外图像，以证明其与传统双线性插值相比的增强性能。
■我们针对彩色和NIR图像进行去马赛克的优化CNN将彩色和NIR的均方误差降低了37％，将NIR的均方误差降低了40％，分别,在临床前数据中，两种成像方式的结构差异指数均提高了37％。在临床数据集中，该网络在彩色图像中将均方误差提高了35％，在NIR图像中将均方误差提高了42％，同时在两种成像方式中将结构相异指数提高了39％。
■我们通过使用为六色图像传感器量身定制的优化CNN，展示了颜色和NIR模式的图像分辨率增强。随着显卡计算能力的不断提高，我们的方法显著提高了分辨率,这对于在手术环境中实时执行是可行的.
■我们引入了一种优化的深度CNN，旨在对使用六色成像传感器获得的彩色和近红外图像进行去马赛克。
■在彩色图像数据集上对残差CNN进行了微调和训练，随后在一系列双通道上进行了评估，颜色,和近红外图像，以证明其与传统双线性插值相比的增强性能。
■我们针对彩色和NIR图像进行去马赛克的优化CNN将彩色和NIR的均方误差降低了37％，将NIR的均方误差降低了40％，分别,在临床前数据中，两种成像方式的结构差异指数均提高了37％。在临床数据集中，该网络在彩色图像中将均方误差提高了35％，在NIR图像中将均方误差提高了42％，同时在两种成像方式中将结构相异指数提高了39％。
■我们通过使用为六色图像传感器量身定制的优化CNN，展示了颜色和NIR模式的图像分辨率增强。随着显卡计算能力的不断提高，我们的方法显著提高了分辨率,这对于在手术环境中实时执行是可行的.
