A big Data Analytics Framework for the Integration of Heterogeneous Federated Data Centers

摘要

Big Data is a collection of data sets that are enormous and complex to store and process using conventional data storing and processing techniques. The emergence of data in different domains causes significant challenges in data manipulation and decision making. In recent years, the requirement for the analysis of heterogeneous data on distributed data storages has been increased and has gained a lot of researchers' attention. With the rapid growth of data, a single cluster environment becomes inadequate to manage this data. At the same time, there are heterogeneous data sources on different platforms, which need to inter-connect to derive meaningful analysis. The MapReduce software paradigm has surfaced to fill the gap, and it has been successfully operating on systems. However, only single cluster environments are supported by the current implementation of MapReduce and this framework cannot be applied to federated heterogeneous data centers. Hence, it does not have enough capabilities to process heterogeneous data sources. This research presents a big data analytic framework that supports the integration of heterogeneous data sources on distributed computing models across multiple data centers. Besides, the performance of the proposed framework is measured under different cluster configurations, and experimental evaluations had shown promising results for the proposed framework compared to a single cluster environment.