A multi-screen refactoring system for video-centric web applications

摘要

The ubiquity of web applications and the multiplication of personal devices are major factors for the increased demand for multi-screen applications. Multi-screen applications impose challenges on the application developer and designer especially if existing single-screen applications have to be transformed to the multi-screen environment. Challenges are related to the user interface division and distribution, layout adaptation, logic re-organization, runtime synchronization and adaptation to the underlying multi-screen platform. This paper faces these challenges and proposes an end-to-end refactoring system. The system allows the re-use of existing single-screen applications to automatically create multi-screen applications. The components of the multi-screen applications have their layout adapted to small and large devices and are ready to operate synchronously to provide a complementary usage experience. Our system is quantitatively evaluated on different sets of applications containing at least one video element and interactive content. Compared to a ground truth, our segmentation approach achieves an average recall of 78 %. Our layout refactoring approach reduces horizontal scrolling by 67 % on the tested applications. Finally, we evaluate the performance of the run-time behavior of one multi-screen application that is highly dynamic, in real physical environment. With a maximum total delay of 34 ms in a LAN, our solution is realistic.