Automation bias with a conversational interface: user confirmation of misparsed information

摘要

We investigate automation bias for confirminguderroneous information with a conversational interface.udParticipants in our studies used a conversational interface toudreport information in a simulated intelligence, surveillance, andudreconnaissance (ISR) task. In the task, for flexibility and ease ofuduse, participants reported information to the conversationaludagent in natural language. Then, the conversational agentudinterpreted the user’s reports in a human- and machine-readableudlanguage. Next, participants could accept or reject the agent’sudinterpretation. Misparses occur when the agent incorrectlyudinterprets the report and the user erroneously accepts it. Weudhypothesize that the misparses naturally occur in the experimentuddue to automation bias and complacency because the agentudinterpretation was generally correct (92%). These errors indicateudsome users were unable to maintain situation awareness usingudthe conversational interface. Our results illustrate concerns forudimplementing a flexible conversational interface in safety criticaludenvironments (e.g., military, emergency operations).