User input and interface design in augmented reality for use in surgical settings

摘要

Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an Interaction Engine. According to various embodiments, the Interaction Engine generates within a unified three-dimensional (3D) coordinate space: (i) a virtual 3D model container; (ii) a virtual 3D medical model positioned according to a model pose within the virtual 3D model container; and (iii) a virtual 3D representation of at least a portion of at least one of a user's hands. The Interaction Engine renders an Augmented Reality (AR) display that includes concurrent display of the virtual 3D medical model container, the virtual 3D medical model and the virtual 3D representation of the user's hands. The Interaction Engine detects one or more physical gestures associated with the user and one or more types of virtual interacts associated with a detected physical gesture(s).