Two-step long short-term memory method for identifying construction activities through positional and attentional cues

摘要

Recognizing construction activities and involved working groups is critical to enhancing construction safety and improving productivity. Most existing studies use videos that only contain one activity with involved entities and rely solely on the spatial-temporal relationship among entities. However, in practice, many workers and machines co-exist and collaborate to accomplish different activities, and not all of them are relevant to the same activity, even though they are spatially close. This paper presents a two-step classification approach - working group identification followed by activity recognition, leveraging both positional and attentional cues, to recognize complex interactions and their involved entities from videos that contain different activities with multiple entities. The spatial and attentional states of individual entities are represented numerically,. and the corresponding positional and attentional cues between two entities are computed. Long short-term memory (LSTM) networks are designed to (1) classify whether two entities belong to the same group, and (2) recognize the activities they are involved in. The newly created method is validated using two sets of construction videos. Identifying working groups before recognizing ongoing activities enables the exclusion of group-irrelevant entities and thus, improves the performance. Moreover, by leveraging both positional and attentional cues, the accuracy increases from 85% to 95% compared with cases using positional cues alone.