The issue of query contention and scalability is the most difficult issue facing organizations deploying real-time data warehouse solutions. The contention between complex se- lects and continuous inserts tends to severely limit the scal- ability of the data warehouses. In this paper, we present a new method called dynamic multi-level caches, to effec- tively deal with the problem of query contention and scal- ability in real-time data warehouses. We differentiate be- tween queries with various data freshness requirements, and use multi-level caches to satisfy these different require- ments. Every query arriving at the system will be automat- ically redirected to the corresponding cache to access the required data, which means that the query loads are dis- tributed to multi-level caches instead of becoming blocked in the only one cache due to the contention between query and update operations. Extensive experiments on several real datasets show that our method can effectively balance the query loads among multi-level caches and achieve de- sirable system performance.
展开▼
