Multiagent Environment Design in Human Computation(Extended Abstract)

摘要

Human computation aims to solve computationally-hard problems, e.g. image tagging or commonsense collection, by utilizing collective human brain power. There are a variety of applications available nowadays. Games with A Purpose (GWAP) [4] engage players in an online game and let them help solve tasks while having fun. Crowdsourcing markets, such as Amazon Mechanical Turk, provide platforms for workers to contribute their brain power in exchange for monetary rewards. Peer productions systems, e.g. Wikipedia or Yahoo! Answers, let online users construct knowledge bases for common good. Despite the impressive progress of developing applications to solve real-world problems, little study is conducted in theory to guide the design of human computation systems, von Ahn and Dabbish [4] discussed the design principles of Games with A Purpose. Some other researchers [3] analyzed the incentive structure of human computation systems in a game theoretic approach. While these projects addressed the design of the system mechanisms, many situations arise when the developers do not have full privilege to modify the systems. For example, developers on Mechanical Turk cannot change the way they interact with the workers. They can only make little modifications, such as the size of payments, or the task descriptions, to encourage workers complete the tasks quickly and accurately. In this work, we focus on situations in which developers can only make limited changes to the systems. In particular, we view this problem as an environment design problem with multiple agents.