LEVERAGING ANNOTATION BIAS TO IMPROVE ANNOTATIONS

摘要

In order to leverage annotation bias in batch annotations, obtained via crowdsourcing, on a set of comments on user posts in a social network, a system may select a subset of the comments for annotation based on how informative expected annotations for the comments in the subset are for the one or more classifiers and probabilities of occurrence of the expected annotations based on a predetermined annotation probability distribution. Note that the classifier may predict how likely the expected annotations are accurate for the comments in a given subset. Moreover, the predetermined annotation probability distribution may specify the annotation bias. In this way, the system may use the annotation bias to select the subset that is likely to receive expected annotations and, thus, are that are easier to use in training the classifier.