Infinite Hidden Conditional Random Fields for the Recognition of Human Behaviour

摘要

While detecting and interpreting temporal patterns of nonverbal behavioral cuesudin a given context is a natural and often unconscious process for humans, itudremains a rather difficult task for computer systems.udIn this thesis we are primarily motivated by the problem of recognizingudexpressions of high--level behavior, and specifically agreement anduddisagreement.udWe thoroughly dissect the problem by surveying the nonverbal behavioral cuesudthat could be present during displays of agreement and disagreement; we discussuda number of methods that could be used or adapted to detect these suggestedudcues; we list some publicly available databases these tools could be trained onudfor the analysis of spontaneous, audiovisual instances of agreement anduddisagreement, we examine the few existing attempts at agreement and disagreementudclassification, and we discuss the challenges in automatically detectingudagreement and disagreement.udWe presentud experiments that show that an existing discriminative graphical model, theud Hidden Conditional Random Field (HCRF) is the best performing on this task. Theud HCRF is a discriminative latent variable model which has been previously shownud to successfully learn the hidden structure of a given classification problemud (provided an appropriate validation of the number of hidden states).udWe show here that HCRFs are also able to capture what makes each of these socialudattitudes unique. We present an efficient technique to analyze the conceptsudlearned by the HCRF model and show that these coincide with the findings fromudsocial psychology regarding which cues are most prevalent in agreement anduddisagreement. Our experiments are performed on a spontaneous expressions datasetudcurated from real televised debates.udThe HCRF model outperforms conventional approaches such as Hidden Markov Modelsudand Support Vector Machines.udSubsequently, we examine existing graphical models that use Bayesianudnonparametrics to have a countably infinite number of hidden states and adaptudtheir complexity to the data at hand.udWe identify a gap in the literature that is the lack of a discriminative suchudgraphical model and we present our suggestion for the first such model: an HCRFudwith an infinite number of hidden states, the Infinite Hidden Conditional RandomudField (IHCRF).udIn summary, the IHCRF is an undirected discriminative graphical model forudsequence classification and uses a countably infinite number of hidden states.udWe present two variants of this model. The first is a fully nonparametric modeludthat relies on Hierarchical Dirichlet Processes and a Markov Chain Monte Carloudinference approach. The second is a semi--parametric model that uses DirichletudProcess Mixtures and relies on a mean--field variational inference approach. Weudshow that both models are able to converge to a correct number of representedudhidden states, and perform as well as the best finite HCRFs ---chosen viaudcross--validation--- for the difficult tasks of recognizing instances ofudagreement, disagreement, and pain in audiovisual sequences.