Factors Influencing the Performance of Human Computation: An Empirical Study in Web Application Testing

摘要

Human computation tackles many difficult-to-automate problems. However, the involvement of human solvers in computation process leads to unstable performance, especially when solvers recruited via Internet. By considering human's attributes and computational situation, the research model of factors of affecting the effectiveness and efficiency of human computation algorithm is proposed in this paper, together with seven proxies of human's attributes and computational situation. For testing the proposed model, we collect data from an implemented Web application testing system using a crowd-sourcing approach. There are 87 volunteers recruited via social network and the corresponding user-session data. The results show that more human solvers with lower ability level, higher engagement, more personal bias and lower difficulty level of tasks lead to shorter completion time. Human solvers with higher ability level, higher engagement, more personal bias and lower difficulty level of tasks lead to more accurate output, but number of solvers does not show significant correlation with the correctness of human computation algorithm.