Predictions of Cloud Storage Trends in 2024

This article, published on December 27, 2023, was written by Eran Kirzner, co-founder and CEO, Lightbits Labs, Ltd.

4 Predictions of Cloud Storage Trends in 2024

Having spent more than a decade serving in tech with roles in engineering management, networking, software, and storage I have a unique perspective on trends in cloud storage. I speak with many business leaders who are shifting their data management strategy and adopting a cloud-first approach, whether it’s the public cloud, on-premises, or hybrid cloud implementation, organizations see the value that cloud architecture delivers in terms of resiliency, efficiency, and pay-as-you-go Opex models.

Industry research supports this trend, with the rate of end-user spending on public cloud services increasing steadily since 2017. Gartner research indicates that CSPs offering specialized platforms will continue to proliferate solving customer challenges by offering customized services tailored to geographic regions. More Gartner research predicts that 35% of data center infrastructure will be managed from a cloud-based control plane by 2027, up from 10% in 2022.

The convergence of technology and market conditions will provide an impetus for the growth of data to be stored on cloud architectures even more next year.

From the proliferation of AI Cloud providers to mandatory cloud storage security features, here are 4 trends that I see rising to the top in 2024:

1. NVMe-oF will gain more momentum as the main tier-1 storage connectivity. ‘Gartner’s storage hype cyclehttps://www.gartner.com/en/research/methodologies/gartner-hype-cycle illustrates NVMe-oF as being 5 to 10 years from organizations fully understanding and realizing the benefits, but in my purview, early adopters are already reaping its benefits to their competitive advantage. The blazing speed of Ethernet and NVMe-oF is a promising cloud storage protocol for today’s performance-sensitive workloads at scale: AI/ML, big data, OLTP, and any other workload that can benefit from high-performance block storage. I predict NVMe-oF will continue to replace iSCSI and FC and become the de facto standard for cloud storage technology and the underlying storage access protocol to support modern-day applications with a thirst for higher performance. It offers cloud-like simplicity with an invisible fabric for a distributed storage network while maintaining enterprise reliability and durability requirements. Industry tech leaders like Microsoft have recognized the convergence of enterprise and modern cloud storage platforms by jumping into the mix and democratizing the NVMe protocol with their announcement at this year’s Microsoft Ignite to support inbox NVMe/TCP–making it available now on all data center OSs.

2. The number and market size of AI cloud providers will grow exponentially. Spanning multiple market industries, organizations are rigorously integrating AI technology into their applications, analytics, business, and services. Mordor Intelligence expects the AI cloud market to grow from $51 billion to more than $207 billion, a CAGR of 32.37% during the forecast period 2023- 2028. This investment is fueling the need for data center modernization and custom AI models (reduced set or segment tailored), and as such we’ll see more native AI services offered by the hyperscalers and a proliferation of AI cloud service providers offering specialized services. AI Cloud service providers with specialized platforms, like Crusoe Cloud, are being launched where the speed and scalability of GPUs, compute, and storage will play a key role in enterprise organization’s successful AI initiatives.

3. Hybrid cloud is here to stay and enabled by software-defined cloud architectures. Hybrid cloud implementations are becoming universal, with organizations using multiple clouds to support diverse application workloads and use cases while delivering business agility. Research supports what I hear from business leaders, that during the forecast period from 2022 to 2028, the hybrid cloud market is predicted to continuously grow at a CAGR of 20.97%. The business leaders that I work with want the flexibility of moving workloads to the cloud platform that offers the best cost-efficiencies, without compromising on performance, scalability, or data services. At the same time, they want the same look, feel, and capabilities from their cloud storage across any deployment platform–on-premises or public clouds. That level of flexibility and portability, the fast provisioning of storage resources to where and when it’s needed, comes with a modern, software-defined cloud architecture. Business leaders with a hybrid- or multi-cloud strategy should prioritize a software-defined architecture that offers license portability across cloud platforms.

4. Data security will be a critical capability of cloud storage systems. Most business leaders that I speak with say their foremost concern with hybrid- or multi-cloud is the complexity of securing their data from intrusion, manipulation, and data sniffing. Gartner validates their concerns with predicted spending on data privacy and cloud security to record their highest growth rates in 2024, with each segment increasing more than 24% year-over-year. Organizations will require detection, protection, and prediction from advanced AI-driven monitoring systems (AIOps) integrated into their cloud storage systems. For many industries (e.g. eCommerce, financial services, and others) encryption at rest and in flight are no longer advanced capabilities, but table stakes for any cloud storage supplier. In addition, many business leaders are moving away from hardware-based encryption with Self-Encrypting Drives and shifting to software-based encryption to reduce storage costs and lead time, eliminate hardware vendor lock-in, and enable cloud and hybrid cloud portability.

Storage is no longer a barrier to executing a cloud-first strategy
Legacy storage appliances with their monolithic architectures are going the way of the dinosaur, incapable of keeping pace with performance-sensitive, cloud-native applications or enabling organization’s cloud-first, hybrid cloud strategies. Public cloud usage is ubiquitous; cloud storage is no longer a barrier to migrating legacy performance-sensitive applications. In tandem, I see many organizations building cloud-native infrastructure within their on-premises data centers and the continued proliferation of CSPs offering specialized platforms. The common thread for successfully building a cloud service is a modern storage system that is software-defined and NVMe-based. This combination delivers unrivaled business benefits offering fast, simple storage provisioning, with the flexibility to move storage services where and when they are needed, as well as lower storage TCO. As we look to the future and to enable better business outcomes, cloud storage systems should deliver cost-efficiency, fast performance, and resiliency.