Graphcore and Pure Storage in Partnership to Optimize Storage Systems for Specific Requirements of AI

To provide performance for AI computing, Graphcore and Pure Storage, Inc. are partnering to optimise storage systems for the specific requirements of AI.

Following the recent addition of storage companies to Elite Partner Program, Graphcorei publishing first reference architecture, in partnership with Pure Storage.

The technical guide details how Graphcore users can take advantage of the Pure’s flexible, performance FlashBlade technology as part of their IPU-POD configuration. The 2 companies are committed to delivering industry performance, while supporting customers’ AI journeys as they transition to next-gen compute systems.

Pure innovation
FlashBlade is an all-flash system, optimized for storing and processing unstructured data, and scalable up to multi-petabyte capacity.

System is based on five innovations:

• Performance storage – all-flash system pairing NAND flash with integrated NVRAM.

• Unified network – consolidating high communication traffic over single network, supporting IPv4 and IPv6 client access over Ethernet up to 1.6Tb/s.

• Purity//FB storage OS – symmetrical OS running on Flashblade’s fabric modules, balancing client operation requests across blades.

• Common media architectural design for files and objects: single underlying media architecture supports concurrent access to files via NFS, NFS over HTTP, SMB, as well as object storage via Amazon S3 protocol.

• Simple usability – Purity//FB on Flashblade performs routine administrative tasks autonomously, self-tuning and providing component failure alerts.

Integration
FashBlade is an all-flash system, optimized for storing and processing unstructured data, and scalable up to multi-petabyte capacity.

IPU-POD64

System is based on five key innovations:
The 2 companies’s reference architecture describes the hosts, storage, and networking configuration used in the IPU-POD64 reference architecture featuring the FlashBlade storage solution.

Click to enlarge

In addition to IPU-POD configuration detailed in RA, FlashBlade storage solution was architected as follows:

• 40x17TB FlashBlade with redundant external fabric modules (XFM)

• External fabric modules connected to Arista Leaf Switches via 8x100GbE configured as a single LACP/MLAG group

Click to enlarge

Performance results
As part of the joint reference architecture, the 2 companies’s benchmarked system performance running ResNet50 and BERT-Large, along with a number of standardised storage benchmarks for AI workloads.

Near-linear scaling was achieved as Graphcore increased the number of jobs run on the infrastructure, while optimal IPU performance was delivered with significant bandwidth capacity remaining on the 40-blade system.

Resources:
Benchmark performance is available in reference architecture document.
Blog: Graphcore adds leading storage providers to Elite Partner Program    
Channel, technology or ecosystem Partner Program page.