Questionnaire Measuring the Utility of Knowledge-Based Systems

摘要

This paper describes the development and validation of an off-the-shelfquestionnaire designed to be tailored, as needed, to obtain the opinions of potential users of knowledge based systems. Development began with a literature review to identify criteria used by different researchers to assess system utility and usability. The identified criteria then were organized into a multi-attributed hierarchy with the top three dimensions being Effect on Task Performance, System Usability, and System Fit. The bottom-level attributes were used to develop the questions for assessing system utility. In May 1994, the questionnaire was successfully tailored and used by the Army's Battle Command Battle Laboratory to evaluate 11 decision aiding prototypes. The questionnaire distinguished between those prototypes the soldiers liked and those that they did not. Psychometric analyses indicated the questionnaire passed required tests for reliability and validity.