A First-Person Mentee Second-Person Mentor AR Interface for Surgical Telementoring

This application paper presents the work of a multidisciplinary group of designing; implementing; and testing an Augmented Reality (AR) surgical telementoring system. The system acquires the surgical field with an overhead camera; the video feed is transmitted to the remote mentor; where it is displayed on a touch-based interaction table; the mentor annotates the video feed; the annotations are sent back to the mentee; where they are displayed into the mentee's field of view using an optical see-through AR head-mounted display (HMD). The annotations are reprojected from the mentor's second-person view of the surgical field to the mentee's first-person view. The mentee sees the annotations with depth perception; and the annotations remain anchored to the surgical field as the mentee moves their head. Average annotation display accuracy is 1.22cm. The system was tested in the context of a user study where surgery residents (n = 20) were asked to perform a lower-leg fasciotomy on cadaver models. Participants who benefited from telementoring using our system received a higher Individual Performance Score; and they reported higher usability and self confidence levels.