Comparative Analysis of Different Text Segmentation Algorithms on Arabic News Stories

摘要

The task of text segmentation represents an important step in many applications and while much work has been carried out to address this task for the English language, work on text segmentation for other languages is still lagging behind. In this paper a comparative analysis of three different text segmentation algorithms on Arabic news stories is presented. To assess how well each algorithm works on Arabic news stories, each was applied on an Arabic Reuters news story dataset and the results were compared. The work in this paper also describes a combination of two of these algorithms that was found to produce better results than any of the presented individual algorithms. It also presents a set of error reduction filters that were found to significantly reduce segmentation errors in the detection of borders in Arabic based news stories.