A Proposed Architecture for Continuous Web Monitoring Through Online Crawling of Blogs

摘要

Getting informed of what is registered in the Web space on time, can greatly help the psychologists,marketers and political analysts to familiarize, analyse, make decision and act correctly based on thesociety`s different needs. The great volume of information in the Web space hinders us to continuouslyonline investigate the whole space of the Web. Focusing on the considered blogs limits our working domainand makes the online crawling in the Web space possible. In this article, an architecture is offered whichcontinuously online crawls the related blogs, using focused crawler, and investigates and analyses theobtained data. The online fetching is done based on the latest announcements of the ping server machines.A weighted graph is formed based on targeting the important key phrases, so that a focused crawler can dothe fetching of the complete texts of the related Web pages, based on the weighted graph.