Standards compliance: Reliability in automated evaluation tools for accessibility.

摘要

With the increased predominance of the Internet in our daily activities, one particular area of concern is content accessibility for people with disabilities. Organizations and Web designers are increasingly aware of the need to make their pages accessible to people with disabilities. They are also increasingly aware of the multiple tools available to automate the validation and repair process to bring their sites into compliance with accessibility standards. This field study assessed the inter-reliability of three such automated evaluation tools used by Web developers to ensure their designs are compliant with Section 508 of the United States Rehabilitation Act of 1973 (as amended by the Workforce Investment Act of 1998), Subpart B Technical Standards, §1194.22 Web-based Intranet and Internet Information and Applications.;The tools are fast, inexpensive and seemingly reasonable solutions to ensure standards compliant designs. However, there are also potential risks associated with relying on these tools. Different tools arrive at different conclusions when assessing the same Web site for errors. Using Krippendorff's Alpha Reliability Co-efficient (Kr-a) as a measure of inter-reliability, the computer-assisted content analysis tested data from 50 Web sites. The report data was analyzed for agreement at both the nominal and ratio levels. For both levels of analysis, results showed that three out of sixteen technical standards were objectively and reliably measured by the automated tools, and those standards with subjective components in their written specifications had the lowest reliability results. Furthermore, those standards related to design rather than technical architecture had the most variance between the tools' reports.