Storage Trends and Predictions for 2016 – StorPool

This article has been published on the blog of StorPool Ltd.

Storage Trends and Predictions for 2016
Post-Flash Storage gets real, software-defined storage gains real traction. 25/50/100GE, AFA, ARM, NVM, HCI, SDS and SMR on the rise

25/50/100GbE
First of the storage trends is for 25/50/100 GbE. The first products have just reached availability, so in 2016 we will see wide adoption of these. For our segment 25 GbE will be especially popular because it is a single-lane technology (like 10G, not like 40G), It is backwards compatible with 10G (uses the same SFP-type connector) and has a low cost for NICs, cables and switches. By the end of the year the cost difference between 10 and 25 will be minuscule, so the choice of 25G should be obvious for any new deployment during 2016.

All-Flash Arrays (AFA)
All-flash memory has been gaining market share during the last few years. In 2016 even late adopters will switch to flash for their primary storage needs. It still has to go through a row or price reduction to become affordable.

ARM servers
ARM servers are coming up with players like Broadcom. AMD and Samsung are also going to try their chances of gaining (share) of the ARM server chip market.

Non-Volatile Memory (NVM)
NVM is a new class of storage media. It is faster than NAND flash, slower than RAM, but it is non-volatile. We expect to see first productized 3D XPoint memory by Intel/Micron. Also HP/SanDisk (now WD) announced cooperation around memristors but it is unsure if we’ll see a product in 2016. The first product will be a 3D XPoint PCIe card to be released in Q1 2016. At the beginning, prices will be higher than the ones of NAND flash, but in the long run, non-volatile memory can replace flash entirely. As a result, NAND flash is going to become the new disk, while NVM will take the role of flash. This means a new crop of storage vendors focusing on the new storage tier, like it happened with NAND flash. We expect first players to be announced in 2016.

Hyper-Converged Infrastructure (HCI)
Hyper-converged solutions are going to get wider adoption as they reduce complexity. However they are still costly and by concept are best suited for smaller deployments of 2-5 nodes and thus for the SMB/ROBO segment. At larger scale HCI is not the solution of choice for cost and manageability reasons.

Hot in the storage trends will be Software-Defined Storage (SDS)
SDS has been in the storage trends and hot topic during last couple of years, however the majority of deployments were PoC/tests. 2016 is the year when SDS becomes widely used in production deployments. The technology has reached a good level of maturity, developed by early adopters and now mainstream companies are going to deploy it in order to get the benefits delivered by software-defined storage and distributed storage.

Shingled Magnetic Recording (SMR) drives
SMR drives will be one of the storage trends of 2016. The economics of this new class of media are clear already – SMR HDDs are nearly half the cost of the closest regular HDD available. Other than that, the attraction of SMR drives is the ability to have an online archive at very reasonable cost rather than an offline tape archive. Also, the software stacks and products incorporating them are just coming into place. They fit between tape and HDDs, at the same time displacing both to a degree. Tape will be dead as a backup technology of the mass market and SMR drives will make it really dead.

New tiers
The NVM and SMR drives will create new tiers of storage. NVM will fit between flash and RAM, displacing the highest end of the flash arrays, and enabling a small segment of new applications. SMR drives will fit between tape and regular HDDs, displacing both to a degree. For me tape is dead now. Adding SMR drives for cold storage to the mix makes it very dead.

Further systems specialization
At the medium scale (100TB-1 PB) there is an increasing realisation that you probably need several different systems for the different needs. The best architecture for an archival system is very different for the best architecture for primary storage. So in 2016 mid-sized storage users will look even more to solve different requirements with different systems.

Death of HDD-only and steep decline of cached-HDD systems
Enterprises finally realize that HDD systems, even when cached and centralized architectures don’t work for their virtualized workloads. We still see quotes by major vendors offering HDD-based systems, where the customer clearly stated they want high performance. This will not fly anymore in 2016. This is all part of the specialization trend: different problems = different solutions.

The pursuit to reduce complexity
Complexity is a major pain point, getting worse with all the latest technologies that companies have to keep up with and manage. Hyper-convergence promises to address these issues to a large degree, but only in the smaller storage deployments. cloud operators also help a lot by hiding the complexity of their infrastructure behind an easy-to-use web interface. On a larger scale – when you have to build a public or a private cloud – the sensible solution is to consider software-defined technologies. Most of them rely on standard hardware (and standard skillset), thus they reduce complexity as well as vendor lock-in. This promises to be one of the huge storage trends in 2016.

Pricing of storage systems changes
It is not about storage anymore – compute network and storage become one integral system. This has considerable impact on how you compare solutions – it is not $/GB and $/IO/s of the storage system anymore. Rather companies have to compare functionality and the total cost of their alternative stacks before choosing.

New approach to backup and DR
There is an increased interest for multi-site deployments. In the larger cloud deployments, the general approach seems to be multiple sites connected over the Internet, with exposed choice to the customer where the cloud should keep backup copies of their data. We don’t see any interest in tape libraries. The modern approach to backup and archiving seems to be large HDDs with wide erasure coding. So backups will be either online or nearly online (withins of powering up the HDDs). We also see increased interest in being able to run applications from the backup system even with decreased performance. The line between backup and DR is blurring.

Storage trends and predictions – Summary
On the highest level there is a paradigm shift from ‘storage as a separate subsystem’ (i.e. stand alone SAN box) to ‘software-defined storage’. Companies will grasp the interplay between applications, compute, network and storage. Storage is becoming an integral part of the entire stack and tightly integrated with network and compute (and applications running on top).

Performance requirements are rising and SSD-level of performance is now the standard. Use cases multiply which drives further specialization of storage systems and new tiers of storage.

With impact lower than the paradigm shift new technologies are adding functionality, but also complexity, which the end users are trying to manage, while innovating to keep their competitive edge.