File and Object Storage at Super Computing 2017

Super Computing 2017 (SC17) was an interesting conference for file and object storage at least to confirm some trends and opinions. We wrote a few days ago a more general recap about the conference.

File Storage
In HPC, we have to split file storage players in two categories: those with their own file system technology whose we can name ‘disruptors’ and those who integrate, optimize and sell Lustre- and IBM Spectrum Scale-based solutions and named ‘incumbents’. You can find some literature about GPFS (General Parallel File System), the former name of Spectrum Scale, renamed in February 2015.

Category 1
Avere Systems, Elastifile, NEC with ScaTeFS, NetApp, Panasas, Pure Storage with FlashBlade, Quantum with StorNext, Qumulo, Quobyte, Rozo Systems and WekaIO. Some of them expose classic file sharing protocols such NFS or SMB with internal specifics and other extends this with their own client/agent software.

All were present at SC17 with dedicated booth excepting WekaIO who had a confidential presence on HPE space, and Rozo Systems, absent as well, who chose to target media and entertainment but the product fits in HPC. HPC and M&E have many similarities.

Quobyte had a reasonable booth for the size of the company and we still wonder how their business is going as the company is extremely confidential.

Panasas is back for a few months with a redesign of its ActiveStor product based now on FreeBSD, a new SMB stack and a desegregated architecture with now dedicated 2U (4 nodes) metadata directors named Director 100. We don’t understand why Gartner, Inc. removed Panasas from its Distributed File Systems and Object Storage Magic Quadrant for the reason that the company “addresses only commercial high-performance computing“.

Category 2
Atos, Cray with ClusterStor acquired from Seagate, DDN, HPE with DDN strategic partnership announcement, IBM, NEC and RAID. Vexata also showed the use of Spectrum Scale with its all flash array VX-100F but it could work with other file systems.

And third category with Ceph-based offering like Suse, Red Hat or Quantum as Rook relies on Ceph.

Special mention for Penguin Computing with FrostByte with currently four SDS solutions available: Red Hat Ceph or Gluster, BeeGFS and Lustre.

Concerning BeeGFS, the German File System is represented and listed two times in the IO-500 list.

The strong presence in HPC is represented by the second category but some of these actors should check several emerging or small players and technologies to see if some partnership will make sense to create differentiators and gain market share.

Some players are hot due to rapid growth and unique technology and here is a group of six attractive players: Avere, Elastifile, Panasas, Qumulo, Rozo and WekaIO. Elastifile is available in Dell EMC catalog, HPE picked DDN and WekaIO. The HPC storage leader, DDN, with Lustre and Spectrum Scale-based systems, would be well inspired to deal with a player such Rozo to provide a range of solutions on their hardware as it would be unique.

Object storage
Object storage was a drama at SC17: only Caringo, a pure object storage player, was present and unveiled Caringo Drive the week before. Cloudian, the leader in on-premise S3 storage, was not there.

Many vendors have shown an object storage product at least on slides or posters as they have one in their product line. It was the case of DDN with WOS, HGST with ActiveScale, IBM with Cloud Object Storage, NetApp with StorageGRID Webscale, Pure Storage with FlashBlade, Quantum with Lattus and Rook or Spectra Logic to name a few.

Fujitsu had a presence but didn’t promote its object storage CD10000 S2 based on Ceph. Same remark for Hitachi Vantara.

Nutanix was present with a small booth and is mentioned here as they announced at their .NEXT Europe conference the week before the conference an object storage service named Acropolis Object Storage Service exposing a S3 interface.

As said, this category was a real disappointment at the show. Does it mean it’s difficult to sustain a business when a company owns and develops only one product especially with the pressure of cloud storage providers, the number of players and S3 de-facto standards with no more differentiators between on-premise solutions? Probably.

In term of uses cases, the conference has confirmed that object storage is a good platform for secondary storage or tier 2 as its main characteristic is its capability to store and protect huge capacity of data. In other words, it has to be controlled via a data management tool and integrated in a multi-tier environment to store large and inactive data coming from primary tier with products from Avere, DDN, Elastifile, Panasas, Quantum, Qumulo, Rozo or WekaIO for instance.

Conclusion
Some storage companies didn’t have a booth but got representation at partners’ booth, probably a lack of finance or even difficulties, a lack of market focus at least, an illustration of their size, so if you don’t see them here, HPC is not their market and they prefer other sectors.

2018 will be an interesting year for file storage, tough for object storage, and anticipate other OEM deals or M&As.