JALI: An Animator-Centric Viseme Model for Expressive Lip Synchronization

摘要

The rich signals we extract from facial expressions imposes highrnexpectations for the science and art of facial animation. Whilernthe advent of high-resolution performance capture has greatly improvedrnrealism, the utility of procedural animation warrants arnprominent place in facial animation workflow. We present a systemrnthat, given an input audio soundtrack and speech transcript,rnautomatically generates expressive lip-synchronized facial animationrnthat is amenable to further artistic refinement, and that is comparablernwith both performance capture and professional animatorrnoutput. Because of the diversity of ways we produce sound, thernmapping from phonemes to visual depictions as visemes is manyvalued.rnWe draw from psycholinguistics to capture this variationrnusing two visually distinct anatomical actions: Jaw and Lip, wherernsound is primarily controlled by jaw articulation and lower-facernmuscles, respectively. We describe the construction of a transferablerntemplate JALI 3D facial rig, built upon the popular facial musclernaction unit representation FACS. We show that acoustic propertiesrnin a speech signal map naturally to the dynamic degree ofrnjaw and lip in visual speech. We provide an array of compellingrnanimation clips, compare against performance capture and existingrnprocedural animation, and report on a brief user study.