R&D: In-Network Packet Processing Architecture for Distributed Storage

2017 IEEE Conference on Network Softwarization (NetSoft) has published an article written by Corey Morrison, and Alex Sprintson, Department of Electrical and Computer Engineering, Texas A&M University.

Abstract: “Distributed file systems enable the reliable storage of exabytes of information on thousands of servers distributed throughout a network. These systems achieve reliability and performance by storing multiple copies of data blocks in different locations across the network. The management of these copies of data is commonly handled by intermediate servers that track and coordinate the placement of data in the network. This introduces potential network bottlenecks, as multiple transfers to fast storage nodes can saturate network links between intermediate servers and storage devices. The advent of open Network Operating Systems and Software Defined Networking presents an opportunity to alleviate this bottleneck, as it is now possible to give network elements enough intelligence to handle bandwidth intensive tasks, such as data replication and reconstruction. In this paper, we propose a new in-network packet processing architecture for distributed file systems. The key idea of the proposed architecture is to offload several actions of distributed file systems onto a new fundamental component of the system that runs on network devices such as routers and switches. We describe the component’s architecture and how it can be integrated into existing distributed file systems. To evaluate the performance of our approach, we implement the proposed solution in a block-level storage array distributed across multiple iSCSI targets. We show that the proposed architecture results in reduced network congestion and a decrease in request latency proportional to the number of storage nodes involved in the request.“