CALBC SILVER STANDARD CORPUS

摘要

The CALBC initiative aims to provide a large-scale biomedical text corpus that containssemantic annotations for named entities of different kinds. The generation of this corpusrequires that the annotations from different automatic annotation systems be harmo-nized. In the first phase, the annotation systems from five participants (EMBL-EBI,EMC Rotterdam, NLM, JULIE Lab Jena, and Linguamatics) were gathered. All anno-tations were delivered in a common annotation format that included concept identifiersin the boundary assignments and that enabled comparison and alignment of the results.During the harmonization phase, the results produced from those different systems wereintegrated in a single harmonized corpus ("silver standard" corpus) by applying a vot-ing scheme. We give an overview of the processed data and the principles of harmo-nization — formal boundary reconciliation and semantic matching of named entities.Finally, all submissions of the participants were evaluated against that silver standardcorpus. We found that species and disease annotations are better standardized amongstthe partners than the annotations of genes and proteins. The raw corpus is now availablefor additional named entity annotations. Parts of it will be made available later on fora public challenge. We expect that we can improve corpus building activities both interms of the numbers of named entity classes being covered, as well as the size of thecorpus in terms of annotated documents.