Investigating Attorney Trust in Machine-enabled Legal Research: A Mixed Methods Approach

摘要

ROSS is a new legal research tool leveraging artificial intelligence. This tool is somewhat unique both interms of its interface and underlying functionality. ROSS’s deviation from more traditional systems mayhave an impact on the adoption of this new technology. The researchers in the current study conducted amixed methods evaluation of ROSS to assess user experience and likelihood of adoption with a particularemphasis on evaluating user trust in the technology. The researchers conducted simulation interviews witheleven experienced bankruptcy attorneys. In addition, a human-computer trust survey was administered toassess user trust in the technology at various time points throughout the study. This quantitative surveymeasured both cognitive and affect-based trust and revealed a substantial increase in trust with exposure tothe system. Participants were particularly impressed with attributes of the system that are believed to formthe cognitive basis of trust, such as the tool’s reliability and understandability. Despite these positivefindings, our qualitative assessment of experiential trust was mixed and most users concluded that theywould not use ROSS as their primary research tool. This decision may be attributed to a lack of faith in thetechnology and concerns related to ROSS’s usability. The simulations were conducted between Februaryand April of 2017. It should be noted that ROSS has been updated since this time and certain findings mayno longer be representative of the current platform.