用于重复充电运营记录的基于块采样的高效聚集查询算法

摘要

The existing query analysis methods usually treat the entity resolution as an offline preprocessing process to clean the whole data set.However,with the continuous increasing of data size,such offline cleaning mode with high computing complexity has been difficult to meet the needs of real-time analysis in most applications.In order to solve the problem of aggregation query on duplicate charged records,a new method integrating entity resolution with approximate aggregation query processing was proposed.Firstly,a block-based sampling strategy was adopted to collect samples.Then,an entity recognition method was used to identify the duplicate entities on the sampled samples.Finally,the unbiased estimation of aggregated results was reconstructed according to the results of entity recognition.The proposed method avoids the time cost of identifying all entities,and returns the query results that satisfy user needs by identifying only a small number of sample data.The experimental results on both real dataset and synthetic dataset demonstrate the efficiency and reliability of the proposed method.%现有查询分析方法通常将实体识别作为线下预处理过程清洗整个数据集,然而,随着数据规模的不断增大,这种高计算复杂性的线下清洗模式已经很难满足实时性分析应用的需求.针对重复充电运营记录上的聚集查询问题,提出一种将近似聚集查询处理与实体识别相结合的方法.首先,通过基于块的采样策略采集样本;然后,在采集到的样本上利用实体识别方法识别出重复的实体;最后,根据实体识别的结果重构得到聚集结果的无偏估计.所提方法避免了识别全部实体的时间代价,通过识别少量样本数据即可返回满足用户需求的查询结果.真实数据集和合成数据集上的实验结果验证了所提方法的高效性和可靠性.