Multimodal Interfaces: Literature Review of Ecological Interface Design, Multimodal Perception and Attention, and Intelligent Adaptive Multimodal Interfaces

摘要

To improve operational effectiveness for the Canadian Forces (CF), the Joint Unmanned Aerial Vehicle Surveillance Target Acquisition System (JUSTAS) project is acquiring a medium-altitude, long-endurance (MALE) uninhabited aerial vehicle (UAV). In support of the JUSTAS project, Defence Research and Development Canada (DRDC) - Toronto is investigating the human factors issues of UAV ground control stations (GCS) interfaces for UAVs and exploring possible solutions using multimodal displays. This report analyzes current literature on multimodal perception and psychology in the context of developing a GCS simulator to evaluate the efficacy of multimodal displays for controlling UAVs. The report discusses the application of Ecological Interface Design (EID) to multimodal interface development, multimodal information presentation in non-visual modalities, and issues and implications of using multiple sensory modalities (e.g. crossmodal effects). In addition, the role of Intelligent Adaptive Interfaces (IAI) with respect to multimodal interfaces and current problems with automation in commercial aircraft are addressed. Recommendations are provided to develop a program of research to enhance the design of GCS interfaces to support future requirements of the JUSTAS project.