Augmented reality-based capture, processing and transfer of occupational knowledge

摘要

An augmented reality, mixed reality or virtual reality device is used to create occupational knowledge documentation, such as instruction and/or repair documentation for industrial equipment or processes. The device can be configured to capture time-synchronized data during a session in which an expert person, wearing or holding the device, performs a task to be documented. The data can include, for example, video data captured by a camera, audio data captured by a microphone within audio proximity of the camera, motion data captured by an inertial measurement unit physically fixed relative to the camera, and spatial data relating to locations of at least one of the plurality of objects within the field of view of the camera, wherein the spatial data is generated by one or more spatial sensors. The captured data can be automatically processed and used to render instruction on an augmented reality, mixed reality or virtual reality device.