HDVM:基于关系矩阵的关联数据压缩查询模型

摘要

随着大数据时代的到来,大量的RDF数据充斥着整个数据网络.RDF(Resource Description Framework)后台引擎管理巨大的数据集时,数据集索引不能全部加载到内存中,导致系统需要执行缓慢的磁盘访问来解决SPARQL查询.本文提出了一种HDVM(Header Dictionary Vector Matrix)压缩查询模型,通过在关联数据集中提取潜在的三元组关系矩阵,以主语向量、谓语向量和宾语矩阵的模型序列化存储来减少关联数据重复出现的次数,允许SPARQL查询在压缩状态下全内存执行.实验结果表明,本文提出的模型比常用的HDT(Header-Dictionary Triples)压缩方式提高了3％-20％的压缩率,同时在三元组个数达到十亿级别的数据集上平均查询时间在400ms左右.%With the arrival of big data era, a large number of RDF(Resource Description Framework) data is flooding the entire Web of Data. Since the indexes of these datasets cannot be fully loaded in main memory when the RDF engines manage these huge datasets, these systems need to perform slow disk accesses to solve SPARQL queries. In this paper, a method named HDVM is proposed to reduce the number of linked data repeated times by extracting the latent triplet relation matrix from the linked dataset, and storing them in the form of subject vector, predicate vector and object matrix, which allows SPARQL queries to be full-in-memory performed without decompression. The experimental results show that the HDVM (Header Dictionary Vector Matrix) model proposed in this paper can improve the compression rate by 3％～20％ compared with HDT(Header-Dictionary Triples), and the query time on billion-level-size dataset reaches average 400 milliseconds.