Tally: A web-based subjective testing tool

摘要

Collecting data from human subjects is a critical and time-consuming part of video quality assessment. In this work, we introduce Tally, a modern web-based tool to help automate data collection for subjective video experiments. It is designed with decoupled voting and viewing interfaces, allowing Tally to be used for both 2D and 3D displays. Its web-based design provides ubiquitous data access, the ability to share and collaborate on projects, and compatibility with mobile devices. Standard ITU test methods are supported, and we are releasing the software as open source for continued development.