PDF(Pubmed)
Abstract：
OBJECTIVE: Up to date, there has been a lack of software infrastructure to connect 3D Slicer to any augmented reality (AR) device. This work describes a novel connection approach using Microsoft HoloLens 2 and OpenIGTLink, with a demonstration in pedicle screw placement planning.
METHODS: We developed an AR application in Unity that is wirelessly rendered onto Microsoft HoloLens 2 using Holographic Remoting. Simultaneously, Unity connects to 3D Slicer using the OpenIGTLink communication protocol. Geometrical transform and image messages are transferred between both platforms in real time. Through the AR glasses, a user visualizes a patient\'s computed tomography overlaid onto virtual 3D models showing anatomical structures. We technically evaluated the system by measuring message transference latency between the platforms. Its functionality was assessed in pedicle screw placement planning. Six volunteers planned pedicle screws\' position and orientation with the AR system and on a 2D desktop planner. We compared the placement accuracy of each screw with both methods. Finally, we administered a questionnaire to all participants to assess their experience with the AR system.
RESULTS: The latency in message exchange is sufficiently low to enable real-time communication between the platforms. The AR method was non-inferior to the 2D desktop planner, with a mean error of 2.1 ± 1.4 mm. Moreover, 98% of the screw placements performed with the AR system were successful, according to the Gertzbein-Robbins scale. The average questionnaire outcomes were 4.5/5.
CONCLUSIONS: Real-time communication between Microsoft HoloLens 2 and 3D Slicer is feasible and supports accurate planning for pedicle screw placement.
METHODS: We developed an AR application in Unity that is wirelessly rendered onto Microsoft HoloLens 2 using Holographic Remoting. Simultaneously, Unity connects to 3D Slicer using the OpenIGTLink communication protocol. Geometrical transform and image messages are transferred between both platforms in real time. Through the AR glasses, a user visualizes a patient\'s computed tomography overlaid onto virtual 3D models showing anatomical structures. We technically evaluated the system by measuring message transference latency between the platforms. Its functionality was assessed in pedicle screw placement planning. Six volunteers planned pedicle screws\' position and orientation with the AR system and on a 2D desktop planner. We compared the placement accuracy of each screw with both methods. Finally, we administered a questionnaire to all participants to assess their experience with the AR system.
RESULTS: The latency in message exchange is sufficiently low to enable real-time communication between the platforms. The AR method was non-inferior to the 2D desktop planner, with a mean error of 2.1 ± 1.4 mm. Moreover, 98% of the screw placements performed with the AR system were successful, according to the Gertzbein-Robbins scale. The average questionnaire outcomes were 4.5/5.
CONCLUSIONS: Real-time communication between Microsoft HoloLens 2 and 3D Slicer is feasible and supports accurate planning for pedicle screw placement.
摘要：
目标：到目前为止，一直缺乏将3D切片器连接到任何增强现实(AR)设备的软件基础设施。这项工作描述了一种使用MicrosoftHoloLens2和OpenIGTLink的新颖连接方法，在椎弓根螺钉放置计划的示范。
方法：我们在Unity中开发了一个AR应用程序，该应用程序使用全息Remoting无线渲染到MicrosoftHoloLens2上。同时,Unity使用OpenIGTLink通信协议连接到3D切片器。几何变换和图像消息在两个平台之间实时传输。通过AR眼镜，用户可视化患者的计算机断层摄影叠加到显示解剖结构的虚拟3D模型。我们通过测量平台之间的消息传递延迟来技术上评估系统。在椎弓根螺钉放置计划中评估了其功能。六名志愿者计划使用AR系统和2D桌面计划器来定位和定向椎弓根螺钉。我们比较了两种方法的每个螺钉的放置精度。最后,我们对所有参与者进行了问卷调查,以评估他们使用AR系统的经验.
结果：消息交换的延迟足够低，可以实现平台之间的实时通信。AR方法不劣于2D桌面计划器，平均误差为2.1±1.4mm。此外，使用AR系统进行的98％的螺钉放置是成功的，根据Gertzbein-Robbins量表。平均问卷结果为4.5/5。
结论：MicrosoftHoloLens2和3D切片器之间的实时通信是可行的，并支持椎弓根螺钉放置的准确计划。
方法：我们在Unity中开发了一个AR应用程序，该应用程序使用全息Remoting无线渲染到MicrosoftHoloLens2上。同时,Unity使用OpenIGTLink通信协议连接到3D切片器。几何变换和图像消息在两个平台之间实时传输。通过AR眼镜，用户可视化患者的计算机断层摄影叠加到显示解剖结构的虚拟3D模型。我们通过测量平台之间的消息传递延迟来技术上评估系统。在椎弓根螺钉放置计划中评估了其功能。六名志愿者计划使用AR系统和2D桌面计划器来定位和定向椎弓根螺钉。我们比较了两种方法的每个螺钉的放置精度。最后,我们对所有参与者进行了问卷调查,以评估他们使用AR系统的经验.
结果：消息交换的延迟足够低，可以实现平台之间的实时通信。AR方法不劣于2D桌面计划器，平均误差为2.1±1.4mm。此外，使用AR系统进行的98％的螺钉放置是成功的，根据Gertzbein-Robbins量表。平均问卷结果为4.5/5。
结论：MicrosoftHoloLens2和3D切片器之间的实时通信是可行的，并支持椎弓根螺钉放置的准确计划。
