A Query Mechanism of Massive Relational Data Cross Regional and Multiple Data Centers

摘要

In this paper, we propose a collaborative query system on large-scale relational data available on multiple nodes in several data centers. This mechanism has a special query engine which can access data across all the nodes. We design and implement a process which is suitable to query tasks characteristics, to ensure efficient implementation of these tasks. The system uses direct data transfer between the query engine and each data center, thus to reduce time consumption caused by data transmission. In addition, we introduce an appropriate data caching mechanism to reserve partial remote data at a local data center and to avoid redundant transmission. Our approach improves the transmission efficiency and reduces the bandwidth requirements for networks and their costs. The test shows that the performance of the proposed system with and higher hit rate of cache data is higher more than the traditional collaborative query method.