Web mining and similarity for documents and databases

摘要

This paper carries out some questions as searching, organizing and indexing documents and databases by content. They are not original questions. The methodologies, which we propose, developed according to the new INTERNET boundaries, are new. INTERNET requires a flexible approach able to take into account the complexity of the information elements in INTERNET and their heterogeneity. The great amount of Web documents, their different types and sources, produces information noise. When users have to search for a document they use a search engine, which returns an ordered list of fragmented documents. Users consider this list or they follow other approaches as well as the methodology proposed by Zamir, that classifies documents to organize results. Clasterization is produced considering fragments produced by the search engine. This paper introduces some basic concepts as the similarity concept between documents/databases. Similarity is the closeness of indexed documents/databases, belonging to the information base, and target document. Closeness is identified according to the characteristics used for indexing a document. Starting from the organization and syntactic structure of web documents described using HTML, we can extract implicit information connected to the lay-out of documents, according to a set of heuristics, which usually emphasize the content information units (i.e. the autonomous information parts of the document). The similarity concept has to take into account: ⅰ) the need to manage indexes produced by different interpretations, which are not necessarily independent; a user (reader) can improve indexes proposed by authors, ⅱ) the need to manage the document presentation according to different users culture. There exists a great number of similarity definitions; for textual information the terminological approach is the most used in the similarity definition. On the other hand, images similarity involves spatial relationships. This paper extends similarity definition byintroducing the concept of content similarity of documents indexes (semantic similarity). Moreover, we extend the similarity concept to Web available databases. It is becoming a web-mining problem involving web documents and metadata of databases accessible by web.