Head-worn 3D-Visualisation of the Invisible for Surgical Intra-Operative Augmented Reality
During surgery surgeons must identify vital anatomical structures (nerves and lymph nodes) to prevent unnecessary healthy tissue damage. Correct intraoperative identification by normal eyesight remains enormously challenging due to natural variation. Therefore, surgeons require high-tech intra-operative imaging for reliable high-resolution visual discrimination of critical anatomical structures. H3D-VISIONAIR offers a breakthrough and disruptive head-worn augmented reality (AR) system for surgeons to see beyond normal eyesight. It consists of two multi-spectral cameras (combining visual range with NIR visualization), a belt computer with real-time data processing, and a high-end stereoscopic head mounted display with wireless connection to the operating room infrastructure. The spectral signature of specific annotated tissues form input to develop machine-learning based models to allow automatic segmentation. The models are used to generate the AR-overlays on top of the normal surgical field of view.