Mobile Interaction Techniques for Large Displays

摘要

Nowadays, large displays are common in our life and work. The dramatic changes in display sizes hinder users' interaction based on traditional input devices. Recently, mobile devices have been massively adopted and have been proposed as interaction devices for large displays. However, how to integrate mobile devices into real world applications involving large displays is limitedly investigated. In our research on using mobile devices, we aim to explore solutions to problems in a large display environment. In this dissertation, a new mobile interaction model has been proposed to allow users to input, manage windows and transfer information in a large display environment. The proposed model consists of three components: MobileInput, MobileWindowManager, and MobileClipboard.;MobileInput enables users to input from their mobile devices to large displays. Working as a mouse, MobileInput can position the cursor on the large display. This is achieved by analyzing keypoint features of both a screenshot of the large display and an image captured by a camera, and calculating mapping relationships. MobileInput simulates mouse events to move the cursor and performs mouse functions. Working as a keyboard, MobileInput provides a scheme to allow users to focus on inputting texts on their mobile devices, and produces those changes to text inputting on the remote display.;MobileWindowManager allows users to access and manage application windows on large displays through their mobile devices. MobileWindowManager provides a local interface for users to launch application windows on large displays. With the introduction of QR codes, users gain access to a target window quickly. Considering the fat thumb problem in mobile interaction, MobileWindowManager provides effective schemes to move and resize windows on the remote display through their personal devices.;MobileClipboard allows users to transfer information within a computer or across computers. MobileClipboard extends mechanisms of system clipboards to support information exchange in multiple device environments. User interface gestures are designed for users to select objects on a display through their cameras. In addition, MobileClipboard designs protocols to support copy and paste procedures among mobile devices and displays.