Mixed reality video production with detached camera

摘要

An unmounted camera, which is used to capture images of a user of a virtual reality application, wirelessly transmits its position, orientation and the camera feed to a computer. The computer determines the location and orientation of the camera in the virtual world and renders a view of the virtual scene from the perspective of the camera. The computer compensates for latency in the camera feed. The user's background is removed from the camera feed and the image of the user is composited with the rendered scene to result in a mixed reality scene. The composited scene is displayed back to the camera operator, either locally or on a separate screen. The system provides the freedom to the camera operator to introduce camera movement in mixed reality video productions.