Development of a Holistic Prototype Hadoop System for Big Data Handling

摘要

In the upcoming smart world where billions of sensors, actuators, communication, and networking devices are expected to be connected with each other, these devices will be of various types. The volume of the data expected to be generated will be huge and due to the heterogeneity of the connected devices, there will be various types of data as well as the rate of data generation will be variable. This leads to a challenge of storage and processing these "Big data". In this paper, development of a Hadoop prototype system is presented. The system is developed using HDFS and MapReduce techniques. Red Hat Linux, Windows 10 with Oracle VM VirtualBox are used as the platform to develop this system.