R&D: Tight Bound of Parallel Request Latency for Erasure-Coded Distributed Storage System

Lecture Notes in Computer Science has published, in ICA3PP 2020: Algorithms and Architectures for Parallel Processing Proceedings, an article written by Xingshun Zou, Wei Li, Xingshun Zou, and Wei Li, Shandong University, Jinan, China.

Abstract: “With big data analysis and cloud storage improving at an astonishing rate in recent years, more and more parallel computing and data analysis techniques are invented and applied to accessing data from a distributed storage system, which has noticeable differences with accessing data from a separate dedicated storage system. The first and the last processes of a data analysis task in a shared filesystem may have a remarkable gap in latency. The latency of parallel file requests from these processes, however, lacks analytical model especially when erasure code is employed as storage strategy. The main contribution of this work is development of a tight bound of parallel file request latency to give a theoretical upper bound of lantency. Additionally, an exact batch file chunk request analytical model in distributed systems adopting erasure code is studied by the technique of two embedded Markov chains, performance measures such as queue length distribution, waiting time distribution, etc. are derived in closed form. Experimental evaluation verifies accuracy of the theoretical upper bound on latency.“