面向复杂数据的对象存储系统

摘要

This paper studies the problem of efficient object store and index in digital library and proposes a data repository system called PuntTable. PuntTable uses a schema-free way to store and get the objects and builds indices to support querying the fields inside the objects. In order to achieve a high throughput and low latency, PuntTable is designed by using multiple content storage engine and index storage engine through two interfaces, PuntStore and PuntIndex. PuntStore and PuntIndex are designed as the storage layer of PuntTable. Both the objects content and their indices can choose the most suitable storage layer for a specific data set. PuntTable is tested and evaluated for the performance of processing object data and index store combination by using varies of data sets with different sin-gle record sizes. These data sets are picked from digital library to simulate the real application scenarios. The result reveals that the proper configuration of storage layer for a particular data set can significantly improve the through-put and drop the latency.%主要研究了数字图书馆应用中数字对象的有效存储和索引机制，提出并设计了针对数字图书馆的数据仓储系统PuntTable。PuntTable使用非关系模式存储和管理对象，并且通过在数据对象内部建立索引来支持查询。PuntTable包括两个主要模块：一个是PuntStore，一种有多存储引擎的数据存储系统；另一个是PuntIndex，一种支持多种索引方式的索引系统。PuntTable实现了高吞吐量和低延迟对象存储，数据对象的索引和内容都可以选择最为合适的存储层来进行存储。使用实际数字图书馆中的数据对PuntTable进行了测试和评估。在测试所用的数据集中，每个数据都采用不同长度，使测试更加接近实际应用。实验结果显示，对于不同的数据集使用不同的存储模型可以显著增大数据库系统的吞吐量，并且有效减少延迟。