A First-Person Mentee Second-Person Mentor AR Interface for Surgical Telementoring

摘要

This application paper presents the work of a multidisciplinary group of designing, implementing, and testing an Augmented Reality (AR) surgical telementoring system. The system acquires the surgical field with an overhead camera, the video feed is transmitted to the remote mentor, where it is displayed on a touch-based interaction table, the mentor annotates the video feed, the annotations are sent back to the mentee, where they are displayed into the mentee's field of view using an optical see-through AR head-mounted display (HMD). The annotations are reprojected from the mentors second-person view of the surgical field to the mentee's first-person view. The mentee sees the annotations with depth perception, and the annotations remain anchored to the surgical field as the mentee moves their head. Average annotation display accuracy is 1.22cm. The system was tested in the context of a user study where surgery residents ( n = 20) were asked to perform a lower-leg fasciotomy on cadaver models. Participants who benefited from telementoring using our system received a higher Individual Performance Score, and they reported higher usability and self confidence levels.