一种基于HBase的数据持久性和可用性研究

摘要

HBase(Hadoop DataBase)是Apache Hadoop项目下的一款非关系型数据库,它是一个基于列簇的开源数据存储系统,关于HBase的研究和应用越来越受到关注。由于HBase会在内存缓存数据后写文件系统,所以缓存的大小成为影响系统性能的一个重要因素。本文提出一种基于备份日志的持久性、可用性方案 Remote Log Process,使得HBase能够在不同的缓存规模获得更好的写性能。实验证明,在保证数据的持久性和可用性前提下, RLP能够在不同的缓存大小下获得稳定的性能,并且在缓存不超过默认设置时明显提高写操作时间性能。%HBase, a NoSql database under Apache Hadoop, is an open source data storage system based on column family. Researches and applications based on HBase is more and more popular. But the size of memory buffer become a key factor to influence system performance as HBase will buffer data in memory before store them on file system. In this paper, we provide a new method based on copied log named Remote Log Process to make HBase perform better on write operation with different buffer size while keeping data durable and available. Experiments result indicates RLP can get a steady performance with different buffer size under the condition to guarantee durable and available of input data, while perform much better than pristine systems if the buffer isn't larger then default value.