DUPLICATION REMOVAL DEVICE, DUPLICATION REMOVAL METHOD AND DUPLICATION REMOVAL PROGRAM

摘要

PROBLEM TO BE SOLVED: To provide a technology for efficiently removing duplication from the N(≥3)th duplicate data.SOLUTION: A duplication removal device includes: first to N(≥3)th Bloom filters (BF); a counting section for successively performing processing of determining whether or not information indicating that data whose content is the same as that of data to be stored exist in a storage device is registered in each of the first to Nth BF until the unregistered BF in which the information is not registered is found or it is found that the information is registered in the Nth BF, and for, when the unregistered BF is found, registering the information indicating that the data to be stored exist in the storage device in the unregistered BF; and a duplication removal section for, when the unregistered BF is found by the counting section, storing the data to be stored in the storage device, and for, when it is found that the information is registered in the Nth BF by the counting section, storing index information for associating the data to be stored with the data with the same content in the storage device in the storage device.