Annotating 2D Imagery with 3D Kinematically Configurable Assets of Construction Equipment for Training Pose-Informed Activity Analysis and Safety Monitoring Algorithms

摘要

The availability of inexpensive and high-quality cameras has enabled research towards computer vision systems for tracking construction productivity and safety by detecting, tracking, and estimating pose of construction resources and recognizing their activities. To verify these algorithms can generalize to novel visual scenes before deployment, large amounts of labelled training and real-world validation samples are needed. While automatic generation of synthetic data helps with the former, obtaining the latter is less practical. To address this gap, we introduce a tool with which designated annotators perform pose annotation by interactively aligning 3D kinematically configurable equipment models with their depictions on 2D images. Multiple types of informative visual annotation can be retrieved using this technique, notably segmentation masks and equipment pose. Experiments demonstrate our tool's effectiveness for annotating long-form videos of earthmoving operations. We demonstrate how our tool can provide ground-truth annotations for the evaluation of a variety of computer-vision algorithms.