METHOD AND SYSTEM FOR GENERATING GROUND TRUTH LABELS FOR AMBIGUOUS DOMAIN SPECIFIC TASKS

摘要

This disclosure relates generally to data processing, and more particularly to a method and system for generating ground truth labels for ambiguous domain specific tasks. The system generates reference data corresponding to a regulation statement being processed, using a crowd sourcing mechanism and then processes the reference data using an Expectation Maximization (EM) model. The EM model determines consensus with respect to ambiguity of terms/phrases, validity of questions, and validity of answers, and then based on the determined consensus, provides questions and answers to disambiguate the regulation statement.