Anticipation of Everyday Life Manipulation Actions in Virtual Reality

摘要

While the comprehension of human actions by computer algorithms is widely used in various disciplines of science and technology, the need to predict the actions before their completion is growing. This prediction allows us to prevent undesirable events and enable an efficient interaction between humans and intelligent systems. Here, we first represent manipulation actions using the Enriched Semantic Event Chain (ESEC) framework which creates a temporal sequence of static and dynamic spatial relations between the objects and next, classify and predict the actions. In this paper, we are interested to compare the predictability power of the ESEC framework with that of human subjects. To this end, we designed an experiment in a virtual reality environment and created 300 video scenarios from 10 every day life manipulations. These While the comprehension of human actions by computer algorithms is widely used in various disciplines of science and technology, the need to predict the actions before their completion is growing. This prediction allows us to prevent undesirable events and enable an efficient interaction between humans and intelligent systems. Here, we first represent manipulation actions using the Enriched Semantic Event Chain (ESEC) framework which creates a temporal sequence of static and dynamic spatial relations between the objects and next, classify and predict the actions. In this paper, we are interested to compare the predictability power of the ESEC framework with that of human subjects. To this end, we designed an experiment in a virtual reality environment and created 300 video scenarios from 10 every day life manipulations. These data were next evaluated by both the framework and 50 human participants. The results were surprising because the framework predicted superior than the humans.