Activity recognition in video sequences over qualitative abstracts of a diagram-based representation schema

摘要

Explicit reasoning over a spatial substrate, i.e., space?time information structures underlying a spatial problem, simplifies reasoning. Diagrammatic reasoning makes use of diagrams for exploiting such underlying structures. This paper proposes a novel approach combining diagrammatic reasoning with qualitative spatial and temporal reasoning techniques to visualize and perceive spatio-temporal relations among objects in a video. The hybrid techniques explore information over the spatial substrate for relational extractions. Different relations among objects in transition define short-term activities. Mealy machines are learned over patterns of short-term activities as activity recognizers. The proposed representation and recognition mechanism is validated by conducting experiments for video activity recognition from DARPA Mind?s Eye and J-HMDB dataset.