A Multimodal Gaze-Controlled Virtual Keyboard

摘要

New portable and noninvasive eye-trackers allow the creation of robust virtual keyboards that aim to improve the life of disabled people who are unable to communicate. This paper presents a novel multimodal virtual keyboard and evaluates the performance changes that occur with the use of different modalities. The virtual keyboard is based on a menu selection with eight main commands that allow us to spell 30 different characters and correct errors with a delete button. The system has been evaluated with 18 adult participants in three conditions corresponding to three modalities: direct selection using a mouse, with the eye-tracker to point at the desired command and a switch to select it, and with only the eye-tracker for command selection. The performance of the proposed virtual keyboard was evaluated by the speed and information transfer rate (ITR) at both the command and application levels. The average speed across subjects was 18.43 letters/min with the mouse only, 15.26 letters/min with the eye-tracker and the switch, and 9.30 letters/min with only the eye-tracker. The later provided an ITR of 44.96 and 57.46 bits/min at the letter and command levels, respectively. The results show to what extent a drop of performance can occur when switching between several modalities. While the speed decreases when controlling the virtual keyboard with the eye-tracker only, the system's performance remains functioning for severely disabled people who have their gaze as one of their only means of communication.