PhD Defence Beerend Gerats | Opening the Third Dimension - Computer Vision and Deep Learning in Surgery with Depth Imaging

Opening the Third Dimension - Computer Vision and Deep Learning in Surgery with Depth Imaging

Beerend Gerats is a PhD student in the Department of Robotics and Mechatronics. Promotors are prof.dr. I.A.M.J. Broeders and dr. J.M. Wolterink from the Faculty of Electrical Engineering, Mathematics and Computer Science.

Surgical practice has undergone a profound transformation with the widespread adoption of minimally invasive and robot-assisted techniques, leading to an increasing reliance on surgical video. Laparoscopic and ambient operating room cameras continuously generate large volumes of data, forming the foundation for surgical computer vision. Although deep learning has accelerated automated analysis of surgical video for applications such as workflow recognition, instrument tracking, and intraoperative assistance, most existing approaches process inherently three-dimensional surgical scenes through two-dimensional projections. This mismatch introduces geometric ambiguity, limits volumetric reasoning, and restricts the robustness of learned representations. At the same time, the privacy-sensitive nature of surgical recordings constrains large-scale data collection that is much needed to advance the research field.

This dissertation investigates the role of depth imaging, three-dimensional scene representations, and geometry-aware deep learning in advancing surgical computer vision. The central hypothesis is that explicitly incorporating volumetric information improves representation quality, enhances downstream task performance, and facilitates privacy-preserving data acquisition.

In the first part of the thesis, neural field–based methods are used to reconstruct continuous three-dimensional representations of intra-abdominal and operating room environments. Its results highlight the feasibility of recovering volumetric structure from conventional surgical imaging. The second part explores explicit geometry-aware deep learning using RGB-D imaging for two applications in the operating room: simulation of pneumoperitoneal deformation and human pose estimation. The studies demonstrate how incorporating three-dimensional structure and symmetry into model design strengthens its robustness. The third part leverages the privacy-preserving characteristics of depth imaging, demonstrating that combining laparoscopic video with operating room depth maps improves workflow recognition and enables estimation of the remaining procedure duration.

Overall, this dissertation establishes that geometry, depth sensing, and volumetric deep learning architectures could play a foundational role in the next generation of surgical computer vision.