Storage Infrastructure Predictions for 2019 – StorPool

The global storage market is expected to be valued at $144.76 nillion by 2022, at a CAGR of 16.76% between 2016 and 2022 and is among the fastest growing technology segments worldwide.

Most of this spend will go to cloud and storage software solutions.

As every year in December, StorPool Ltd. makes a review of what happened throughout the year and releases its annual predictions. Here it summarizes the most important trends and changes we saw in the storage market in 2018 and list our predictions for 2019.

Hybrid future
The widespread adoption of the cloud is obvious. There is a lot of noise around ‘100% cloud’ strategies, yet enterprise IT spending on cloud is just 19% in 2018 and forecasted to grow to 28% in 2022, according to Gartner. This includes SaaS offerings too and even in 3 to 4 years, less than a third of money spent will be in the cloud.

Now, that being said: hybrid cloud architectures will pick up the pace in 2019. AWS and Azure have strong hybrid strategies and there is a number of third-party vendors providing solutions to manage multi-cloud and hybrid cloud infrastructure and to provide underlying services such as storage and networking for hybrid cloud.

With Amazon’s release of Outposts (now in preview), IT organizations, system integrators, and solution providers will be forced to consider their public and hybrid cloud strategy.

It has to be noted, though, that this is predominantly unstructured data, like videos, photos and new-age cloud native applications and SaaS. The majority of core storage services are still kept locally, with burst or backup/DR being the usual off-load to the cloud.

For more demanding workloads and sensitive data, on-premise is still king. I.e. the future is hybrid: on-premise takes the lead in traditional workloads and cloud storage is the backup option; for new-age workloads, cloud is the natural first choice and on-prem is added when performance, scale or regulation demands kick-in.

From legacy SANs to software-defined storage
5 years ago, the adoption of modern software-defined storage (SDS) solutions, capable of replacing a high-end SAN or AFA was in its early days. Now the SDS ecosystem has grown and matured.

2018 was the year when we saw mainstream enterprises to finally initiate projects to replace traditional SANs solutions. The most common drivers are the need for agility, cost optimization and performance improvements, which are needed to meet the changing and increasing business demands. We expect SDS adoption to have a spillover effect and gain majority market share over the next 3 to 5 years.

Infrastructure refresh cycles and performance complaints from customers/users are the top two triggers of this process. Investments in new-generation infrastructure software solutions are aimed at reducing vendor lock-in, achieving cost optimizations and accelerating application performance.

Use of SDS, especially at the high end of the performance spectrum (NVMe, NVMeOF) and when it comes to automation through APIs and integrations, is the only meaningful way to differentiate public and private cloud services, especially at scale.

FC is dead
At this point FC is becoming an obsolete technology and we do not see neither financial, nor performance justification to deploy FC in an IT infrastructure stack. Additionally FC adds complexity in an already complex environment, being a separate storage-only component.

In 2019, it makes sense to deploy a parallel 25GbE standard Ethernet network, instead of upgrading an existing FC network. At scale, the cost of the Ethernet network is 3-5% of the whole project and a fraction of cost of a FC alternative.

100GbE is becoming the typical network connectivity for demanding environments.

NVMeoF and NVMe/TCP will have a gradual increase in adoption. At the low latency end of the spectrum, they will still be considered the second-best option, after proprietary access protocols (with storage driver in the initiator host).

Next-gen storage media
Persistent memory in the form of DRAM-based NVDIMMs finally became widely available on the market in 2018. We expect next-gen storage media to gain wider adoption in 2019. Its primary use-case will still be as cache in SDS systems and database servers.

On a parallel track, Intel will release large capacity Optane-based NVDIMM devices, which they are promoting as a way to extend RAM to huge capacities, at low cost, through a process similar to swapping. The software stack to take full advantage of this new hardware capability will slowly come together in 2019.

There will be a tiny amount of proper niche usage of persistent memory, where it is used for more than a very fast SSD.

In the same way, as it happened with SSDs and then flash-based NVMes, storage solutions will struggle to expose the high throughput and low latency of the new (persistent memory) media to applications. So, as usual – be wary of marketing-defined storage slideware, stressing hyped buzz-words, void of reasonable application.

ARM in the datacenter
ARM is (finally) being recognized as a serious potential alternative to the x86 server architecture, for particular workloads. The main drivers here are cost optimization and breaking vendor lock-in.

ARM is still not fast enough to compete for general purpose workloads, but in 2018 we saw the first CPUs which were fast enough to be a serious contender for owning a solid piece of the server CPU market. The recently announced AWS instances, powered by Amazon’s custom Arm based CPU, which claim up to 45% cost savings, will definitely pave the way to a wider ARM adoption.

The prime use case for ARM servers in 2018 was ‘ARM test/dev’, which is self explanatory. In 2019 we’ll see raising demand for ARM, however this will still be a slow pickup, as wider adoption requires the proliferation of a wider ecosystem.

Throughput-driven, batch processing workloads in the datacenter and small compute clusters on ‘the edge’ are the two prime use-cases for ARM-based servers for 2019.

The multi-core race
Intel and AMD are on a race to provide high core-count CPUs for servers in the datacenter and in HPC. AMD announced its 64-cores EPYC 2 CPU with overhauled architecture (9 dies per socket vs EPYC’s 4 dies per socket). At the same time, Intel announced its Cascade Lake AP CPUs, which are essentially two Xeon Scalable dies on a single (rather large) chip, scaling up to 48 cores per socket. Both products represent a new level of per-socket compute density. Products will hit the market in 2019.
While good for the user, this is ‘business as usual’ and not that exciting.

Global IT, storage and infrastructure market changes
In our last year’s predictions, we wrote that we expect a wave of consolidations in 2018. While these are a natural process in the world of business, the tectonic shifts that happen in IT infrastructure are reflected here.

There were several high-profile acquisitions. Most of them were not directly storage related, yet they gave out signals of the massive transformation of the IT infrastructure landscape.

The more notable acquisitions, were:

• Microsoft acquiring Github, interesting because of the increasing involvement of Microsoft with developers, Open Source and Linux communities.
• IBM buying Red Hat for $33 billion, which was a bit of a surprise. It is interesting to see how the largest open source company in the world will ‘behave’ as it changes the color of its hat to blue.

The rumored bid of Microsoft for Mellanox is definitely worth mentioning here, as Azure is the only real contender to the undisputed cloud leader, Amazon AWS.

On the storage market, Tintri filed for bankruptcy in the US under Chapter 11 in the summer and was then acquired by DataDirect Networks (DDN).

Somewhat overlooked, as not as shiny were a myriad of regional deals between 2nd and 3rd tier cloud providers. Data centers and cloud providers shifted workloads to newly opened/acquired local facilities in bigger markets, so they can save and serve data locally.