Construction of gazetteers from geo big data using machine learning technique on Hadoop

摘要

Most gazetteers have been built and maintained for the purpose of visualizing geographical location on the Geographical Information system (GIS) client. The advent of big data allows us to construct gazetteers by directly mining rich volunteered information from the web. In this, we propose a technique for extracting location based spatial information from the web documents and media services like flickr, twitter, facebook for construction of gazetteers. To achieve this, we need to search the web for existing data pertaining location. A web crawler (Google search engine) generates the web pages based on the location keyword given by the user and maintaining the index of the web pages and the proposed system passes it to the Hadoop environment. For further simplification, the name node transfers the index group of web pages to different data nodes for extraction of spatial information from the dynamic web documents that we gather using machine learning process. Each data node is then utilized for the generation of a common template. The common template allows the extraction of location based spatial information from the dynamic web documents and media services. Resultant information from the data node is further merged using map reduce algorithms and the Hadoop Distributed File System (HDFS) is produced which is then converted to Geo-Java Script Object Notation (JSON) format, thus aiding in the task of visualizing the extracted information on the GIS client.