Spatial-Temporal Bottom-Up Top-Down Attention Model for Action Recognition

摘要

Driven by the importance of capturing non-local information in video understanding, we propose Spatial-temporal Bottom-up Top-down Attention Module (STBTA). Features are processed across in multiple scales and then combined to best capture the spatial relationships associated with the region of interest and the surrounding environment in a complicated scene. Attention maps are used for adaptive feature refinement. STBTA can be plugged into any feedforward network architectures and is end-to-end trainable along with CNN. Extensive experiments on UCF101, HMDB51, Kinetics-400 datasets demonstrate that the proposed method can improve the performance for action recognition.