An IO Optimized Data Access Method in Distributed Key-Value Storage System

摘要

Distributed KEY-VALUE storage system is a new storage framework for cloud computing. It can enable an application to dynamically adapt to growing workloads by increasing the number of servers. However, current distributed KEY-VALUE storage systems are still inefficient on range query for larger result set. When the result set become large, the file layout, cache hit rate are both key points for IO efficiency. In this paper, we will introduce our experience under the development of China Mobile Big Cloud KEY-VLAUE DB (BC-kvDB). We will discuss how we increase IO efficiency in BC-kvDB. BC-kvDB is based on single-table space data model and provides SQL-LIKE DDL and DML language. BC-kvDB's high throughput is benefit from data locality storage, column-storage structure and multi-layer caches. Data can be accessed in local cache or local blocks through block index. Experimental results show that the random writing performance of BC-kvDB is 2.5 times better than HBase and the random reading performance is 1.8-2 times than HBase.