Improvement WSD Dictionary Using Annotated Corpus and Testing it with Simplified Lesk Algorithm

摘要

WSD is a task with a long history in computational linguistics. It is open problem in NLP. Thisresearch focuses on increasing the accuracy of Lesk algorithm with assistant of annotatedcorpus using Narodowy Korpus Jezyka Polskiego (NKJP “Polish National Corpus”). TheNKJP_WSI (NKJP Word Sense Inventory) is used as senses inventory. A Lesk algorithm isfirstly implemented on the whole corpus (training and test) and then getting the results. This isdone with assistance of special dictionary that contains all possible senses for each ambiguousword. In this implementation, the similarity equation is applied to information retrieval using tfidfwith small modification in order to achieve the requirements. Experimental results show thatthe accuracy of 82.016% and 84.063% without and with deleting stop words respectively.Moreover, this paper practically solves the challenge of an execution time. Therefore, weproposed special structure for building another dictionary from the corpus in order to reducetime complicity of the training process. The new dictionary contains all the possible words (onlythese which help us in solving WSD) with their tf-idf from the existing dictionary with assistantof annotated corpus. Furthermore, eexperimental results show that the two tests are identical.The execution time - of the second test dropped down to 20 times compared to first test withsame accuracy.