Patterns of evidence: Investigating information in visible speech perception.

摘要

The goal of this investigation was to find the information in a speaker's face used in visible speech perception. Three hypotheses were also tested: (a) information is statistically independent and not just orthogonal; (b) independent features are more functional than orthogonal features, six visible features, or linguistic features; (c) multiplicative integration of the FLMP better describes information processing than additive integration (AMP) or non-integration (RACE).; At the outset, information was clearly defined and five components of information were identified: (a) Empirical; (b) Functional; (c) Contextual; (d) Independent; and (e) Minimal. These five components are criteria that must be satisfied to concluded that information has been uncovered in any inquiry. To meet all five criteria a General Framework of Perception (GFP) was proposed as a basis for a General Model of Perception (GMP). GMP meets all the criteria by linking models of sensory encoding, Independent Components Analysis (ICA), with models of information processing, Multidimensional Fuzzy Logical Model of Perception (MD-FLMP).; ICA was trained on images of a speaker saying 22 English consonants in three vowel contexts. The resulting features of ICA and several competing theories were then fit against subject responses using MD-FLMP, AMP, and RACE. Responses were obtained from an experiment in which subjects viewed the same speaker saying one of 264 monosyllabic English words without sound and identified the word that was spoken. Responses for the initial CV of each word were collected and organized in a consonant confusion matrix.; Results showed that information comes from areas of the face not typically explored in studies of visible speech; namely, chin movement independent of jaw rotation, neck flexation, and skin warping/wrinkling. Information was also found from several well-known facial regions including the lips, teeth, and jaw. Several lip configurations were functional including rounding, unrounding, spreading, compression, and protrusion. Model tests confirmed all three hypotheses supporting the functionality of the independent features and the multiplicative integration assumption of the MD-FLMP over additive integration and non-integration. Applications of this approach to human speech training and a visible speech recognition system were discussed.