Improving the verification process of video quality metrics

摘要

The most important step in the development process of a video quality metric is its verification with regards to the subjective quality experience. Even though guidelines in the form of standards and recommendations are well known, there are still quite often shortcomings in the verification process of many metrics. In this contribution we revisit these rules, point out important details and review contributions to video quality metrics for typical shortcomings. We will highlight in detail five steps that should be followed in order to improve the overall quality of the verification process of video quality metrics: using a large and diverse data base, planing and conducting subjective tests carefully, using different data for calibration and verification of a metric, avoiding unnecessary data fitting steps, and a clear and meaningful presentation of the results. Also we will provide short examples how an improper verification might affect the results of a video quality metric.