High-Performance Inline De-Dupe for Non-Volatile Memory

NV-de-dupe saves NVM space, also boosts performance of PMFS by up to 2.1x.
This is a Press Release edited by on 2017.12.01

AddThis Social Bookmark Button

IEEE Transactions on Computers as published an article written by Chundong Wang, Engineering Product Development Pillar, Singapore University of Technology and Design, 233793 Singapore, Singapore, Qingsong Wei, Networks Storage, Data Storage Institute, Singapore, Singapore, Jun Yang, Cheng Chen, Yechao Yang, and Mingdi Xue, Data Center Technologies, Data Storage Institute, Singapore, Singapore.

Abstract: "The byte-addressable non-volatile memory (NVM) is a promising medium for data storage. NVM-oriented file systems have been designed to explore NVM's performance potential. Meanwhile, applications may write considerable duplicate data. For NVM, a removal of duplicate data can promote space efficiency, improve write endurance, and potentially improve the performance by avoidance of repeatedly writing the same data. However, we have observed severe performance degradations when implementing a state-of-the-art inline deduplication algorithm in an NVM-oriented file system. A quantitative analysis reveals that, with NVM, 1) the conventional way to manage deduplication metadata for block devices, particularly in light of consistency, is inefficient, and, 2) the performance with deduplication becomes more subject to fingerprint calculations. We hence propose a deduplication algorithm called NV-Dedup. NV-Dedup manages deduplication metadata in a fine-grained, CPU and NVM-favored way, and preserves the metadata consistency with a lightweight transactional scheme. It also does workload-adaptive fingerprinting based on an analytical model and a transition scheme among fingerprinting methods to reduce calculation penalties. We have built a prototype of NV-Dedup in Persistent Memory File System (PMFS). Experiments show that, NV-Dedup not only substantially saves NVM space, but also boosts the performance of PMFS by up to 2.1x."