Towards computer-vision software tools to increase productionand accessibility of video description for people with vision loss

摘要

This paper presents the status of a R&D projecttargeting the development of computer-vision tools toassist humans in generating and rendering video descrip-tion for people with vision loss. Three principal issues arediscussed: (1) production practices, (2) needs of peoplewith vision loss, and (3) current system design, coretechnologies and implementation. The paper provides themain conclusions of consultations with producers of videodescription regarding their practices and with end-usersregarding their needs, as well as an analysis of describedproductions that lead to propose a video descriptiontypology. The current status of a prototype software isalso presented (audio-vision manager) that uses manycomputer-vision technologies (shot transition detection,key-frame identification, key-face recognition, key-textspotting, visual motion, gait/gesture characterization, key-place identification, key-object spotting and image catego-rization) to automatically extract visual content, associatetextual descriptions and add them to the audio track with asynthetic voice. A proof of concept is also briefly describedfor a first adaptive video description player which allows endusers to select various levels of video description.