A Summary Evaluation Method Combining Linguistic Quality and Semantic Similarity

摘要

Summary evaluation method is crucial to promote the development of text summarization technologies. However, most of the existing summary evaluation methods seldom consider the content integrity and readability of summaries simultaneously. This paper proposed a Linguistic Quality and Semantic Similarity Model (LQSSM) to evaluate generated summaries more comprehensively. Considering the readability of summaries, a linguistic quality evaluation network (LQEN) is proposed to evaluate summaries automatically. Meanwhile, a semantic similarity evaluation network (SSEN) is introduced to directly measure the informativeness between the summary and original text. Additionally, there is also a comprehensive evaluation using fusion indicators. The LQSSM and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) methods were used to score the summaries generated by five models. The results showed that the proposed method gave further feedback from different angles on the basis of reaching the level of other methods.