Evaluating Robot-Operator Job Performance

摘要

Assessments of small air and ground robot systems often focus on how well the equipment functions. Those assessments also should include the human operator and how well the robot and operator work together as an integrated system. Tests that do include performance of the human operator often rely on qualitative observations -- observer judgments and interviews about workload, situation awareness, cognitive issues, and so on. This paper views the operator and robot as a team, outlines a schema for measuring robot-operator team performance, and presents an initial proof-of-principle test for quantitatively assessing that performance. The initial proof-of-principle test did the following: (1) defined robot-operator performance factors associated with moving a small robot from point A to point B, and (2) quantified the effects that different sensor and navigation technologies have on that performance. The proof-of-principle approach was to instrument the robot and the operator interface to allow measures of operational performance in navigation- reconnaissance tasks. The instrumentation enabled measurement of efficiency and effectiveness (e.g., frequency of control actions, time between control actions) and errors and accuracy. These kinds of measurements provide data about mission performance contributions spanning robot capabilities, operator skills, employment strategy, interface limitations, and so on. The test demonstrated the usefulness of evaluating robot-operator teams to assess success and failure factors in the performance of robots and the effects of different technologies on that performance.