Bengali Stop Phrase Detection Mechanism using Corpus Based Method

摘要

This paper discusses a corpus-based method for the detection of the stop phrase. These phrases must be detected and eliminated during NLP in the quest for attaining efficient indexing in modern Information Retrieval (IR) systems. A complete set of stop phrases for the Bengali language has not been developed yet. In this paper, a corpus-based approach is introduced for recognizing and extracting Bengali stop phrases. This proposed technique indicates that an input paragraph will be tokenized in several required manners and after that identification of stop phrases will be obtained by checking through the corpus. Accepted stop phrases will be sent for uniqueness. Outcomes of this proposed approach for stop phrases detection are notable where accuracy, precision, and recall results are observable. Eliminating these stop phrases will further reduce the time complexity of those algorithms, which were used in case of text summarizing and IR system.