An Improved Human-Object Interaction Detection Method Based on Short-term Memory Selection Network

摘要

Human-object interaction (HOI) detection task is defined as inferring all the 〈 human, verb, object 〉 triplets in the image, which helps computers to obtain a more comprehensive understanding of the visual scene. Most existing HOI detection methods focus on instance local features, and rarely consider the information from backgrounds. Our core idea is that the relationship between human, object and other backgrounds contains important cues to facilitate HOI detection. According to the short-term memory selection (STMS) mechanism, we regard the interaction relationship as the result of human and object stimulating the union area, and simulate the stimulation process by the recurrent neural network. The features in the union area of human and object are taken as the input of RNN, human and object are the two inputs of RNN, and the output is the representation of the interaction relationship. Combined with the visual features and spatial features of instances, a multi-stream network is utilized to detect HOIs in the image. Experiments on V-COCO and HICO-DET show that the proposed model achieves better performance, verifying the effectiveness of our method.