An Efficient Inline Data Deduplication with Data Relationship Manager for Cloud Storage

摘要

In the current digital and cloud world, the data handling is a biggest challenge for all the service providers. One way or other way we need more storage backup system to handle the disaster recovery. For the high performance, the video and audio streaming application, database application, multimedia applications need more memory and high speed process. Using data deduplication concept in the storage system, we can reduce memory space requirement and can improve system performance. This paper is focusing on Metadata management and Replacement Algorithm to provide the high throughput and minimal resource utilization. By implementing LIRS replacement algorithm in the deduplication system for caching, it reduces the IO access compare to other replacement algorithm. Also this system improves the LIRS Meta data lookup speed using Data Relationship Manager. We have explained the complete Architecture, Write and Read process, Algorithm implementation and discussed the different experiment results. In our study the LIRS with Data Relationship Manager improves the time taken for the deduplication and reduces the IO access. We compared 4000 files with 5 different data pattern, the result shows the LIRS with Data Relationship Manager works well on the weak locality data pattern. The result clearly shows that without relationship manager, LIRS is taking much more time for the deduplication. The Time and Workload pattern comparison shows the result is improved and it reduces the memory access when we have Data relationship manager. LIRS with Data relationship manager improves the deduplication efficiency and throughput.