Abstract：
OBJECTIVE: Patient-to-image registration is a preliminary step required in surgical navigation based on preoperative images. Human intervention and fiducial markers hamper this task as they are time-consuming and introduce potential errors. We aimed to develop a fully automatic 2D registration system for augmented reality in ear surgery.
METHODS: CT-scans and corresponding oto-endoscopic videos were collected from 41 patients (58 ears) undergoing ear examination (vestibular schwannoma before surgery, profound hearing loss requiring cochlear implant, suspicion of perilymphatic fistula, contralateral ears in cases of unilateral chronic otitis media). Two to four images were selected from each case. For the training phase, data from patients (75% of the dataset) and 11 cadaveric specimens were used. Tympanic membranes and malleus handles were contoured on both video images and CT-scans by expert surgeons. The algorithm used a U-Net network for detecting the contours of the tympanic membrane and the malleus on both preoperative CT-scans and endoscopic video frames. Then, contours were processed and registered through an iterative closest point algorithm. Validation was performed on 4 cases and testing on 6 cases. Registration error was measured by overlaying both images and measuring the average and Hausdorff distances.
RESULTS: The proposed registration method yielded a precision compatible with ear surgery with a 2D mean overlay error of 0.65 ± 0.60 mm for the incus and 0.48 ± 0.32 mm for the round window. The average Hausdorff distance for these 2 targets was 0.98 ± 0.60 mm and 0.78 ± 0.34 mm respectively. An outlier case with higher errors (2.3 mm and 1.5 mm average Hausdorff distance for incus and round window respectively) was observed in relation to a high discrepancy between the projection angle of the reconstructed CT-scan and the video image. The maximum duration for the overall process was 18 s.
CONCLUSIONS: A fully automatic 2D registration method based on a convolutional neural network and applied to ear surgery was developed. The method did not rely on any external fiducial markers nor human intervention for landmark recognition. The method was fast and its precision was compatible with ear surgery.
METHODS: CT-scans and corresponding oto-endoscopic videos were collected from 41 patients (58 ears) undergoing ear examination (vestibular schwannoma before surgery, profound hearing loss requiring cochlear implant, suspicion of perilymphatic fistula, contralateral ears in cases of unilateral chronic otitis media). Two to four images were selected from each case. For the training phase, data from patients (75% of the dataset) and 11 cadaveric specimens were used. Tympanic membranes and malleus handles were contoured on both video images and CT-scans by expert surgeons. The algorithm used a U-Net network for detecting the contours of the tympanic membrane and the malleus on both preoperative CT-scans and endoscopic video frames. Then, contours were processed and registered through an iterative closest point algorithm. Validation was performed on 4 cases and testing on 6 cases. Registration error was measured by overlaying both images and measuring the average and Hausdorff distances.
RESULTS: The proposed registration method yielded a precision compatible with ear surgery with a 2D mean overlay error of 0.65 ± 0.60 mm for the incus and 0.48 ± 0.32 mm for the round window. The average Hausdorff distance for these 2 targets was 0.98 ± 0.60 mm and 0.78 ± 0.34 mm respectively. An outlier case with higher errors (2.3 mm and 1.5 mm average Hausdorff distance for incus and round window respectively) was observed in relation to a high discrepancy between the projection angle of the reconstructed CT-scan and the video image. The maximum duration for the overall process was 18 s.
CONCLUSIONS: A fully automatic 2D registration method based on a convolutional neural network and applied to ear surgery was developed. The method did not rely on any external fiducial markers nor human intervention for landmark recognition. The method was fast and its precision was compatible with ear surgery.
摘要：
目的：患者到图像的配准是基于术前图像的手术导航所需的初步步骤。人为干预和基准标记会妨碍这项任务，因为它们耗时且会引入潜在的错误。我们旨在开发一种用于耳部手术中增强现实的全自动2D配准系统。
方法：收集了41例（58耳）接受耳部检查的患者（术前前庭神经鞘瘤，严重的听力损失需要人工耳蜗植入，怀疑淋巴瘘,在单侧慢性中耳炎的情况下，对侧耳）。从每种情况中选择两到四个图像。对于培训阶段，我们使用了来自患者(数据集的75%)和11例尸体标本的数据.由专业外科医生在视频图像和CT扫描上对鼓膜和锤骨手柄进行了轮廓绘制。该算法使用U-Net网络在术前CT扫描和内窥镜视频帧上检测鼓膜和锤骨的轮廓。然后，通过迭代最近点算法对轮廓进行处理和配准。对4例进行了验证，对6例进行了测试。通过重叠两个图像并测量平均和Hausdorff距离来测量配准误差。
结果：所提出的配准方法产生了与耳部手术兼容的精度，圆窗的2D平均叠加误差为[公式：见文字]mm，[公式：见文字]mm。这两个目标的平均Hausdorff距离分别为[公式：见文本]mm和[公式：见文本]mm。与重建的CT扫描的投影角度和视频图像之间的高度差异有关，观察到具有较高误差的异常值情况（砧木和圆窗的平均Hausdorff距离分别为2.3mm和1.5mm）。整个过程的最大持续时间为18s。
结论：开发了一种基于卷积神经网络并应用于耳部手术的全自动2D配准方法。该方法不依赖于任何外部基准标记或人为干预来识别地标。该方法快速，精密度与耳部手术相符。
方法：收集了41例（58耳）接受耳部检查的患者（术前前庭神经鞘瘤，严重的听力损失需要人工耳蜗植入，怀疑淋巴瘘,在单侧慢性中耳炎的情况下，对侧耳）。从每种情况中选择两到四个图像。对于培训阶段，我们使用了来自患者(数据集的75%)和11例尸体标本的数据.由专业外科医生在视频图像和CT扫描上对鼓膜和锤骨手柄进行了轮廓绘制。该算法使用U-Net网络在术前CT扫描和内窥镜视频帧上检测鼓膜和锤骨的轮廓。然后，通过迭代最近点算法对轮廓进行处理和配准。对4例进行了验证，对6例进行了测试。通过重叠两个图像并测量平均和Hausdorff距离来测量配准误差。
结果：所提出的配准方法产生了与耳部手术兼容的精度，圆窗的2D平均叠加误差为[公式：见文字]mm，[公式：见文字]mm。这两个目标的平均Hausdorff距离分别为[公式：见文本]mm和[公式：见文本]mm。与重建的CT扫描的投影角度和视频图像之间的高度差异有关，观察到具有较高误差的异常值情况（砧木和圆窗的平均Hausdorff距离分别为2.3mm和1.5mm）。整个过程的最大持续时间为18s。
结论：开发了一种基于卷积神经网络并应用于耳部手术的全自动2D配准方法。该方法不依赖于任何外部基准标记或人为干预来识别地标。该方法快速，精密度与耳部手术相符。
