Spatial relations for integrating visual images of physical environment into virtual reality

摘要

A method for use with a head mounted display in a physical environment includes obtaining depth information of the physical environment and capturing a visual image of the physical environment. The method also includes determining a spatial relationship between a user of the head mounted display and one or more physical objects contained within the physical environment based on the depth information. The visual image is then segmented based on spatial relationships to generate a segmented image that includes one or more physical objects. The segmented image is then overlaid on the virtual image to display both the virtual image and one or more physical objects on the head mounted display.