Identifying and removing duplicates in DataStream applications is one of the primary challenges in traditional duplicate elimination techniques. It is not feasible in many streaming scenarios to eliminate precisely the occurrence of duplicates in an unbounded data stream. However, existing variants of the Bloom filter cannot support dynamic in both filter and counter together. In this paper we focus on eliminating the duplicates by introducing the dynamic approach on both the size of the counter and the bloom filter. The basic idea is instead of keeping either the size of counter or filter static in this paper we improvised the performance of by considering both the counter and the filter size as dynamic. In addition necessary algorithms for new item insertion, querying on the membership and deleting the duplicates are also proposed. we showed that a the proposed approach guarantees the superiority in terms of accuracy and time efficiency and reduces the considerable amount of false possible rate than the existing approaches.
展开▼
