Learning Multi-level Interaction Relations and Feature Representations for Group Activity Recognition

摘要

Group activity recognition is an challenging task with a major issue that reasons about complex interaction relations in the context of multi-person scenes. Most existing approaches concentrate on capturing interaction relations and learning features of the group activity at individual or group levels. These approaches lose sight of multi-level structures and interaction relations of the group activity. To overcome this challenge, we propose a Multi-level Interaction Relation model (MIR) to flexibly and efficiently learn multi-level structures of the group activity and capture multi-level interaction relations in the group activity. MIR employs graph pooling and unpooling networks to build multi-grained group relation graphs, and thus divide the group activity into multiple levels. Specifically, the Key Actor based Group Pooling layer (KeyPool) selects key persons in the activity to build the coarser-grained graph while the Key Actor based Group Unpooling layer (KeyUnPool) reconstructs the finer-grained graph according the corresponding KeyPool. Multiple KeyPool and KeyUnPool progressively build multi-grained graphs and learn multi-level structures of the group activity. Thanks to graph convolutions performed on multi-grained relation graphs, multi-level interactions are finally captured. In addition, graph readout (GR) layers are added to obtain multi-level spatio-temporal features of The group activity. Experimental results on two publicly available datasets demonstrate the effectiveness of KeyPool and KeyUnPool, and show our model can further improve the performance of group activity recognition.