Challenges in the analysis of multimodal messaging

摘要

New forms of computer-mediated communication are increasingly multimodal, providing capabilities for communicating with some combination of text, image, audio, and video. In this paper, we point to the need to develop better methods for studying multimodal communication -- more specifically, for studying thencommunicative role of and relationships among differentnmodalities within their increasingly complex, multimodal semioticnlandscapes. We present two challenges in the analysis of multimodal communication, point of view and unit of analysis, both encountered in the context of our study of the use of photo-enhanced instant messaging.