Attention Estimation for Input Switch in Scalable Multi-display Environments

摘要

Multi-Display Environments (MDEs) have become commonplace in office desks for editing and displaying different tasks, such as coding, searching, reading, and video-communicating. In this paper, we present a method of automatic switch for routing one input (including mouse/keyboard, touch pad, joystick, etc.) to different displays in scalable MDEs based on the user attention estimation. We set up an MDE in our office desk, in which each display is equipped with a webcam to capture the user's face video for detecting if the user is looking at the display. We use Convolutional Neural Networks (CNNs) to learn the attention model from face videos with various poses, illuminations, and occlusions for achieving a high performance of attention estimation. Qualitative and quantitative experiments demonstrate the effectiveness and potential of the proposed approach. The results of the user study also shows that the participants deemed that the system is wonderful, useful, and friendly.