METHOD AND DEVICE FOR EXTRACTING CORE WORDS FROM COMMODITY SHORT TEXT

摘要

The present invention discloses a method and a device for extracting core words from commodity short text, which relates to the field of big data processing. Wherein the method for extracting core words from commodity short text comprises: acquiring the commodity short text in a data set; performing word segmentation processing for the commodity short text; obtaining a document vector of the commodity short text according to context information of the word segmentation of the commodity short text; clustering the commodity short text in the data set on the basis of the document vector; determining the clustering hierarchy weight of each word segment in the commodity short text in a category to which the commodity short text belongs; and determining core words of the commodity short text according to the clustering hierarchy weight of each word segment. The present invention refers to the context information of the word segment in the commodity short text to obtain the document vector of the commodity short text, which can make up the shortcoming of small information amount of the short text, make the clustering result based on the document vector more accurate, and thereby extract core words more accurately from the commodity short text according to the weight of the word segment in the clustering category to which the commodity short text thereof belongs.