SYSTEM FOR DE-DUPLICATING NETWORK CODED DISTRIBUTED STORAGE AND RELATED TECHNIQUES

摘要

Methods for encoding a file and de-duplicating a coded packet of the file are presented. The encoding method includes: identifying a file to be stored; encoding the file to generate a plurality of coded packets; generating a hash code associated with the file; distributing the plurality of coded packets to a plurality of storage nodes; and storing the hash code associated with the file in a hash server. The de-duplicating method includes: receiving a coded packet at a storage node; retrieving a hash code, from the hash server, corresponding to the coded packet; determining whether the coded packet has been stored in the storage node based upon the hash code; in response to determination that the coded packet has being stored, discarding the coded packet; and in response to determination that the coded packet has not being stored, storing the coded packet in the storage node.