TUNING CROWDSOURCED HUMAN INTELLIGENCE TASK OPTIONS THROUGH FLIGHTING

摘要

Different options associated with the performance of intelligence tasks are flighted. Different versions of applications that provide the context within which intelligence tasks are performed are sourced for each of the different combinations of options. Subsets of the human workers are selected and provided such different versions of such applications. Correlations are made between those applications that were selected by individual workers and the options that such applications represented, and further between intelligence task results, including, optionally, an evaluation of the quality of such task results, and such options. Options, and the settings thereof, which affect and optimize the intelligence task results generated by workers, are, thereby, more efficiently identified.