基于Hive的海量web日志分析系统设计研究

摘要

A massive Web logs analysis mechanism based on Hive is proposed in this paper to solve the bottleneck of traditional mass data processing and calculation. With the Hadoop distributed system architecture and the Hive data warehouse, massive web logs are analyzed and researched, so are the users' browsing behaviors. The analysis of page views, bounce rate, IP number, and section heat ranking in users browsing behaviors provides a guiding light for website building and optimization of large data analysis system.%针对于传统对海量数据处理和计算的瓶颈,提出了一种基于Hive的海量web日志分析机制.通过Hadoop分布式系统架构以及Hive数据仓库对海量web日志做了分析处理,并对用户的浏览行为进行了分析研究.对用户浏览行为中的浏览量和跳出率、IP数、版块热度排行的分析结果对于网站建设和大数据分析系统优化都具有一定的指导意义.