Evaluation Methodology for Natural Language Processing Systems.

摘要

The Neal-Montgomery NLP Evaluation Methodology was developed under the 'Benchmark Investigation/Identification' project as a means of determining the linguistic competence of Natural Language Processing (NLP) systems. Embodied in an evaluation tool based on a detailed classification of linguistic phenomena with, currently, over 350 test items, the methodology produces descriptive profiles of NLP system linguistic capabilities, and can be applied without regard to the current system application, domain, or specific language processing function. To date, it has been applied to three NL data base query systems and three text processing systems. Researchers are invited to apply the evaluation methodology (included within this report) to their systems and report their critique and recommendations towards the evolution of standard evaluation procedures for NLP systems.... Natural language processing, Computational linguistics, Evaluation, Benchmark.