Local 3D Reconstruction and Augmented Reality Visualization of Free-Hand Ultrasound for Needle Biopsy Procedures

摘要

We have developed a 3D ultrasound augmented reality (AR) workspace, in which the user wears a video-see-through head-mounted display to observe both 2D ultrasound (US) images and local 3D ultrasound volumes in-situ, i.e. in their actual location, overlaid onto a stereoscopic video view. In this unified platform, the user has the ability to reconstruct a local three-dimensional volume from a series of scanned two-dimensional US B-planes. In-situ visualization of rendered three-dimensional volumes facilitates the understanding of the object's three-dimensional structure. The user has full control over acquisition, reconstruction, and visualization of the volumetric data through an interactive, intuitive augmented reality user interface. Furthermore, the system has the ability to track a hand-held tool (e.g. biopsy needle). Graphically enhanced needle improves the perception of needle's position and orientation with respect to the lesion target in the rendered volume. In phantom experiments, our AR system has proven to effectively facilitate ultrasound guided needle biopsies. We have also verified biopsy results by reconstructing the "post-operative" volumes with the needle left in place.