Updating Annotator Collections Using Run Traces

摘要

Methods, systems, and computer program products for updating annotator collections using run traces are provided herein. A computer-implemented method includes generating one or more alternate versions of one or more document annotators selected from a set of multiple document annotators; executing, on one or more document data sets, (i) one or more document annotators from the set of multiple document annotators and (ii) the one or more alternate versions to generate log information for each document annotator in the set and each alternate version of the one or more alternate versions; and outputting an instruction to modify, based on the generated log information for each document annotator in the set and each alternate version, at least one document annotator from the set with at least one alternate version from the one or more alternate versions.