Advantages and Disadvantages of Remote Asynchronous Usability Testing Using Amazon Mechanical Turk

摘要

Amazon’s Mechanical Turk is an online service that connects people all over the world who arewilling to work for monetary compensation with companies and individuals who need simple taskscompleted that are best done by humans. This paper examines the benefits and caveats from the use ofMechanical Turk as a platform to conduct an online usability test. Our findings indicate thatMechanical Turk can be a very useful tool when it is employed to collect basic user generated datasuch as click time and location, text input, and subjective usability data. In situations where morecomplex usage data is required, Mechanical Turk can still be a valuable resource to run a pilot test toassist in determining potential usability problems. However, in its current form, Mechanical Turk haslimitations and can only be used to assess websites; it cannot be relied upon to conduct a usability teston software without the use of a virtual machine.