Persistent Storage Solutions for Kubernetes-Based Applications

This market report, published on July 18, 2023, was written for Gigaom by Joep Piscaer, technologist with team building and tech marketing skills, and background as a CTO, cloud architect, infrastructure engineer and DevOps culture coach having built many engineering and architect teams and culture, founder of TLA Tech, a tech marketing firm focusing on cloud-native having blogs at https://www.tla-tech.io/joeppiscaer/.

Persistent Storage Solutions for Kubernetes-Based Applications
GigaOm Radar for Cloud-Native Kubernetes Storagev4.0

1. Summary
The adoption of cloud-native, container-based architectures and application modernization continue to fuel demand for persistent storage on Kubernetes platforms. Organizations understand that the benefits of cloud-native workloads in terms of performance, scalability, and portability are key enablers for achieving business goals.

Many enterprises already run cloud-native workloads and realize the advantages of more agile and flexible architectures, including application portability that enables frictionless workload movement from the data center to the cloud, and even across clouds. This provides greater flexibility and responsiveness to business requirements than using legacy technologies.

Storage solutions for Kubernetes environments have evolved since our last report, especially in the realm of migration and mobility, as well as in security features for maturing enterprises, advanced data services, and an enhanced developer experience.

A common pattern in adopting persistent storage solutions for Kubernetes is the reuse of existing enterprise storage solutions. This is considered a safe bet for the first couple of deployments, but these systems weren’t designed with the ephemeral nature of containers in mind. Often, older arrays can’t cope with the sheer number of backend operations required by Kubernetes at scale. However, vendors are quickly removing bottlenecks from their architectures to support containers at scale and stretching their product portfolio to include cloud storage services for multicloud use cases.

Compared to other types of storage systems, Kubernetes-native storage offers a more DevOps-friendly environment, helping to build a hardware stack that can be controlled by the operations team while enabling developers to allocate and monitor resources quickly when necessary. This is a major boon for enterprise IT organizations looking for the smartest way to evolve their processes and align them with the latest business and technology requirements.

Organizations can now consider more factors than ever before when choosing where their applications and data should run-and they want the freedom to decide where that should be. The public cloud is known for its flexibility and agility, but on-premises infrastructures are still better in terms of efficiency, cost, and reliability.

With widespread adoption across cloud, edge, and on-premises, Kubernetes is instrumental in executing the vision of portable, flexible, and agile hybrid cloud strategies, making applications and their data portable and cloud-agnostic-for the most part. It needs the right integration with infrastructure layers-such as storage-to complement its still-maturing native support for stateful storage.

It’s still a significant task to select and implement a Kubernetes storage solution for persistent data that makes the most of Kubernetes’s application mobility and data portability capabilities.

With Kubernetes now supporting business-critical applications and services, requirements have become more stringent. Scalability, performance, resilience, security, and other non-functional requirements are the order of the day, and Kubernetes needs to do it all to ensure a consistent level of throughput without service disruptions. These requirements drive the demand for enterprise stateful data services, solid security controls, mature multitenant performance management-like QoS and bandwidth throttling-and thorough alerting, reporting, and monitoring.

Lastly, enterprises do not want to be locked into any single vendor’s ecosystem as they reap the benefits of Kubernetes’s portable and agnostic promise, so they’re looking for a storage solution that works with feature parity across on-premises and cloud infrastructures.

This GigaOm Radar for cloud-native Kubernetes storage will focus on Kubernetes-native storage systems. These storage solutions are built as cloud-native microservices, running on top of and tightly coupled with the container orchestrator while providing storage services to Kubernetes clusters, including the cluster they are deployed on. These storage systems allow organizations to deploy Kubernetes-optimized persistent storage capabilities to Kubernetes clusters and are mostly suited for new projects or the process of replacing general-purpose enterprise storage systems. These are architectures specifically designed to address the needs of cloud-native applications without compromising on performance or scalability. They are usually not engineered to co-exist with other workload types, such as virtualization.

This is our 4th year evaluating the Kubernetes storage space in the context of our Key Criteria and Radar reports.

All solutions included in this Radar report meet the following table stakes-capabilities widely adopted and well implemented in the sector:

• CSI compatibility
• Snapshot functionality
• Operational and data security
• Cloud and platform support

This report highlights key cloud-native Kubernetes storage vendors and equips IT decision-makers with the information needed to select the best fit for their business and use case requirements. In the corresponding GigaOm report Key Criteria for Evaluating Kubernetes Storage Solutions, we describe in more detail the capabilities and metrics that are used to evaluate vendors in this market.

2. Market Categories and Deployment Types
In this report, we’re evaluating Kubernetes-native storage, referring to solutions built specifically to support stateful containers with scalable, distributed architectures. Typically, the storage system itself runs as a set of containers on a Kubernetes cluster, exposing storage via container storage interface (CSI) to the cluster to be consumed by workloads, and runs alongside the application workloads in the Kubernetes cluster.

These distributed storage solutions are tightly coupled with the container orchestrator and are container-aware so that when the orchestrator spins up or destroys a container, it also handles storage provisioning and deprovisioning operations. Storage operations are automated and invisible to the user.

These solutions are built to recognize and solve the challenges of Kubernetes storage and thus seamlessly integrate with the container ecosystem. The architectures have the tightest integration with the container environment; they closely follow and implement new technologies and protocols developed to extend Kubernetes storage capabilities. They also provide the best performance in day-to-day usage.

These solutions also scale more easily, adhering to the autoscaling rules of the cluster. If a cluster node is added or removed, the storage system automatically scales up and down as well. This automation makes this type of storage very flexible and dynamic, closely aligning with the application design paradigms it supports. Often, solutions in this category use storage policies to decouple workloads from the physical storage media, and they are hardware-agnostic, supporting a wide range of commodity servers and cloud services without any fundamental adjustments.

Note that GigaOm is publishing a separate Radar report on Kubernetes storage focused on general-purpose enterprise storage systems that support Kubernetes-based container environments. Enterprise Kubernetes storage allows organizations to leverage existing deployed storage platforms to deliver persistent storage capabilities without having to architect new solutions. These sol,ons are mostly suited to mixed-workload environments or large data centers with a sizable investment in storage infrastructure.

To better understand the market and vendor positioning (Table 1), we assess how well solutions for cloud-native Kubernetes storage are positioned to serve specific market segments and deployment models:

• SMB: In this category, we assess solutions on their ability to meet the needs of organizations ranging from small businesses to medium-sized companies. Also assessed are departmental use cases in large enterprises where ease of use and deployment are more important than extensive management functionality, data mobility, and feature set.
• Large enterprise: Here, offerings are assessed on their ability to support large and business-critical projects. Optimal solutions in this category have a strong focus on flexibility, performance, data services, and features to improve security and data protection. Scalability is another big differentiator, as is the ability to deploy the same service in heterogeneous environments, including on-premises and cloud. Finally, the developer experience is weighed in this category because large enterprises often need self-service capabilities for their development teams.
• Independent service provider/MSP (ISP/MSP): In this category, solutions that are suitable for ISPs and MSPs are assessed. These should include additional security and multitenancy capabilities and the ability to throttle performance per tenant.

In addition, we recognize 2 deployment models for solutions in this report:

• Traditional enterprise storage: These are controller-based physical hardware arrays, often with hardware optimizations to reduce power consumption, physical space, and increase performance. These solutions serve block, file, and sometimes object storage.
• Kubernetes-native storage: These are hyperconverged, software-only architectures that run as container-based applications. They leverage the Kubernetes scheduler and other capabilities to serve up block (or file) storage to containers running on the same or another Kubernetes cluster, and utilize each compute node’s storage capabilities.

Key to a successful deployment is a solution’s ability to go where the data goes. In other words, it’s important to determine whether the storage solution can be deployed on-premises, in the cloud, at the edge, and at smaller ISPs. Such flexibility takes the solution’s architecture into account and indicates whether it can be deployed easily across the variety of environments organizations have to cope with.

Table 1. Vendor Positioning

+++ Exceptional: Outstanding focus and execution
++ Capable: Good but with room for improvement
+ Limited: Lacking in execution and use cases
– Not applicable or absent

3. Key Criteria Comparison
Building on the findings from the GigaOm report, “Key Criteria for Evaluating Kubernetes Storage Solutionshttps://research.gigaom.com/report/key-criteria-for-evaluating-kubernetes-data-storage-solutions/,” Table 2 summarizes how each vendor included in this research performs in the areas we consider differentiating and critical in this sector. Table 3 follows this summary with insight into each product’s evaluation metrics-the top-line characteristics that define the impact each will have on the organization.

The objective is to give the reader a snapshot of the technical capabilities of available solutions, define the perimeter of the market landscape, and gauge the potential impact on the business.

Table 2. Key Criteria Comparison

+++ Exceptional: Outstanding focus and execution
++ Capable: Good but with room for improvement
+ Limited: Lacking in execution and use cases
– Not applicable or absent

Table 3. Evaluation Metrics Comparison

+++ Exceptional: Outstanding focus and execution
++ Capable: Good but with room for improvement
+ Limited: Lacking in execution and use cases
– Not applicable or absent

By combining the information provided in the tables above, the reader can develop a clear understanding of the technical solutions available in the market.

4. GigaOm Radar
This report synthesizes the analysis of key criteria and their impact on evaluation metrics to inform the GigaOm Radar graphic in Figure 1. The resulting chart is a forward-looking perspective on all the vendors in this report, based on their products’ technical capabilities and feature sets.

The GigaOm Radar plots vendor solutions across a series of concentric rings, with those set closer to the center judged to be of higher overall value. The chart characterizes each vendor on 2 axes-balancing Maturity vs. Innovation, and Feature Play vs. Platform Play-while providing an arrow that projects each solution’s evolution over the coming 12 to 18 months.

Figure 1. GigaOm Radar for Cloud-Native Kubernetes Storage

As you can see in the Radar chart in Figure 1, the cloud-native Kubernetes storage space is evolving rapidly, solutions are innovative, and the market response is dynamic. Especially compared to last year, there are many changes to the vendors included in this report and their go-to-market strategies. This scenario also explains why, like in the last few years, there are no vendors in the Maturity half of the Radar.

In the Innovation/Platform Play quadrant at the bottom right are vendors that are building cloud-native storage platforms. These are vendors that see persistent storage as their unique differentiation and are building a product portfolio around it.

Common in this group is the (optional) coupling of a storage platform with a Kubernetes distribution and cluster management product, the combination of which creates a highly integrated turnkey solution for customers looking to make their entrance into the world of Kubernetes. In this group, we see the most complete feature sets from the strongest contenders in this Radar, each with strong enterprise approaches, mature advanced data services, and well-executed developer experiences.

In the Innovation/Feature Play quadrant at the lower left are two intertwined vendors undergoing a major strategy shift. Since IBM took over Red Hat’s storage portfolio, its product development has integrated multiple, sometimes overlapping, technologies, and it’ll take some time before IBM will have cleaned house. It remains to be seen if there will be cloud-native storage solutions from IBM that are available to other customers besides Red Hat OpenShift customers.

5. Vendor Insights

DataCore
Its OpenEBS PRO is a commercially supported derivative of OpenEBS Mayastor, which it acquired in 2021. OpenEBS PRO is a proprietary, enterprise-grade cloud-native storage solution for Kubernetes, based on the Open core of the OpenEBS Mayastor code base but positioned and built as a turnkey product to overcome Mayastor’s inherent complexities, such as its plug-in system and community-controlled roadmap.

OpenEBS PRO’s differentiation from Mayastor is its ease of use, both in terms of deployment and operations. It’s aimed at developer and DevOps users, not storage admins, broadening its applicability compared to OpenEBS Mayastor. Note that OpenEBS, the open-source project at the base of Mayastor, also remains available.

OpenEBS PRO’s hyper-converged, containerized architecture allows it to scale with the application and takes care of node resilience by replicating volume data across nodes in a cluster. Reads are spread across replicas for optimal performance. Its Intel SPDK-based architecture is very well suited to high-performance, low-latency stateful applications. OpenEBS PRO is software-only and runs on on-premises hardware, as well as on multiple cloud platforms, including AWS, Azure, and GCP.

OpenEBS PRO is a new entrant to the market and is missing some data services, including asynchronous replicas and clones, although support for clones is scheduled to be released soon, and snapshot replication is on the 2024 roadmap. It currently supports full-copy backups and Incremental volume snapshot backups at the block device level. Thin provisioning was added in the May 2023 release.

While the company is expected to add crucial missing features, this gap raises the question of whether customers should choose OpenEBS PRO or stay with Mayastor for the time being to enjoy features like application-consistent snapshots, data-at-rest encryption, and data optimization. Interestingly, DataCore is seeing traction with cloud providers adopting OpenEBS Mayastor as the underpinning technology for some cloud storage services.

Strengths: A turnkey derivative of Mayastor, OpenEBS PRO has potential to be a performance-oriented solution for companies without dedicated storage admins. Its roadmap is looking strong.

Challenges: Recently abandoned the previous incarnation of this product, Bolt, opting to drive net-new features into OpenEBS Mayastor. It’s also missing a few enterprise capabilities (such as data protection and replication).

OpenEBS PRO hasn’t seen an update to the product in over a year, and it is still missing many enterprise capabilities (such as data protection, replication, and footprint optimization). OpenEBS PRO will need substantial effort from DataCore to reach feature parity compared to the competition.

Diamanti
It offers solutions consisting of Kubernetes cluster management (Spektra Enterprise) and Kubernetes data storage (Ultima Enterprise), with an optional physical appliance containing a hardware I/O accelerator card (Ultima Accelerator). It targets enterprise-grade, stateful application use cases.

Ultima Enterprise is its software-only, hyperconverged data plane that converges networking and storage, can run on-premises or in the cloud, and has various deployment options. The Ultima data plane consists of a distributed storage platform that also provides layer 2 and layer 3 networking capabilities, data protection features, container and VM support, and CNI/CSI plug-ins.

The solution also comes with enterprise-grade features. Data can be mirrored across availability zones. Basic crash-consistent snapshots, backup and restore, and disaster recovery (with recovery and fire drill workflows) are supported across clusters and clouds; volumes can be migrated across clouds using asynchronous replication. Notably, these migration features do not require Ultima storage on both source and target environment, which increases migration flexibility.

The firm supports role-based access control (RBAC) and multitenancy (with Spektra), allowing policy-based isolation between tenants and teams. Those features, along with its QoS support, are a plus for MSPs considering delivering Kubernetes as a service to their clients. Data-at-rest encryption is supported at the volume and disk levels, and an advanced, built-in key management system is also provided.

The vendor has a feature-rich management platform that allows organizations to manage multiple clusters across various clouds. It embeds cluster and application lifecycle management capabilities to enable faster application deployments. The management platform also integrates granular observability capabilities, providing an overall view of the environment’s health status and digging all the way down to the container level. Its GroundWork Monitor increases Diamanti’s monitoring and observability capabilities.

Spektra is the container management plane that enables management of Kubernetes clusters across clouds and locations (including core and edge), adding application and data mobility features, plus advanced data services, infrastructure observability, and control. Additionally, OpenShift is a factory-supported deployment possibility for customers who are more comfortable with that option, and it comes with Ultima storage underneath.

Strengths: NVMe-based hyper-converged architecture delivers high resilience and good performance. The combination of its software-only deployment models and flexible data migration features make Diamanti a great data mobility and edge solution.

Challenges: Data protection and data reduction features are lagging behind the competition.

IBM
It delivers cloud-native Kubernetes storage capabilities through IBM Storage Fusion (previously named Spectrum Fusion), a software-defined solution designed for OpenShift. With the integration of Red Hat’s storage portfolio into the IBM Storage business unit, its Fusion product is undergoing a transformation. Currently, the product consists of Scale, Protect Plus, and IBM Fusion Data Foundation (which is similar to Red Hat’s OpenShift Data Foundation Advanced and has the same feature set as ODF as part of OpenShift Platform Plus). Fusion adds some additional value on top of ODF, like a single global namespace and transparent data placement over different storage tiers, as well as policy-based data protection.

Storage Fusion is a cloud-native architecture delivering policy-based storage to OpenShift customers. Its strength is separating storage consumption (including more advanced data services) by developers from storage management by Kubernetes admins via policies, which are highly integrated into OpenShift. It offers block, file, and object services, as well as data protection features (by bundling IBM Storage Protect). It also includes application-aware disaster recovery capabilities, plus support for data migration use cases, and offers data efficiency capabilities in the form of erasure coding support.

Storage Fusion can leverage existing enterprise storage systems, including non-IBM block storage. Storage Fusion is optionally available as an integrated hardware appliance, based on IBM Spectrum Scale, and its software-only deployment supports both on-premises and cloud environments.

The solution is managed using the Storage Fusion HCI dashboard, which provides standard monitoring and alerting capabilities. Integrations are possible with IBM Cloud Satellite and OpenShift Advanced Cluster Management. Storage Fusion also includes call-home support and troubleshooting capabilities.

An interesting feature of the solution is the availability of application packs, which consist of ready-to-deploy packages for popular applications such as Cassandra, Kafka, MongoDB, and SAP HANA.

Strengths: Offering is a Kubernetes storage solution specifically designed to easily deploy the Red Hat OpenShift container platform in hyperconverged mode and has significant benefits for AI/ML workloads on the platform.

Challenges: Storage Fusion is exclusive to OpenShift platforms, inhibiting broader adoption for non-OpenShift users, and data services are not as advanced as its enterprise storage lineup. With IBM’s integration of Red Hat’s storage portfolio in late 2022, the future of both IBM’s and Red Hat’s storage products are unclear.

Portworx by Pure Storage
Portworx is one of the most advanced solutions for cloud-native Kubernetes storage. PX-Store is a hyperconverged, Kubernetes-native storage solution, aggregating and pooling storage capacity for cluster consumption. A series of advanced data management components that are part of the Portworx Data Services platform delivers more advanced storage capabilities, including database lifecycle management.

The solution offers broad deployment choices and supports bare metal and virtualized environments, including Pure Storage physical arrays, existing cloud block services, and cloud-based Kubernetes services, as well as those from other ecosystem partners, providing a consistent experience across infrastructures, platforms, and locations.

Portworx includes a comprehensive set of advanced data services. Portworx Data Services’ database-as-a-service platform is a unique capability, automating the lifecycle of database provisioning and deployment, Day 2 operations, and data protection with support for MongoDB, Elastic, Cassandra, MySQL, Kafka, ZooKeeper, PostgreSQL, RabbitMQ, CouchBase, Consul, and Redis. Support for Microsoft SQL Server as a new data service was added in 2023.

PX-Backup handles data protection and supports application-consistent backups that are Kubernetes-complete, so not only is the data backed up, but so is the entire application state, including all objects, application configuration data, and dependencies. Granularity is also provided, allowing organizations to back up either individual applications or thousands of applications and namespaces, and to define schedule policies as required. Restores can be performed locally or on any cloud.

PX-Store is a modern, distributed, container-optimized cloud-native storage solution with elastic scaling, storage-aware class-of-service, multiwriter shared volumes, local and cloud snapshot capabilities, and multiple failover options (node aware, rack aware, availability-zone aware). Local synchronous replication for data center resilience is also supported. With PX-FAST and a new version of PX-Store, Portworx is starting to natively leverage NVMe performance. Scale-out object storage is an early access feature and will be compliant with COSI when it becomes generally available. This object service allows for creating, claiming, and managing object storage buckets on Amazon S3 or Pure Storage FlashBlade arrays.

PX-DR (an add-on license) expands those capabilities to provide disaster recovery and data replication capabilities. It supports multisite synchronous replication and zero RPO disaster recovery within a metro area, and multisite asynchronous replication for cross-WAN connections. Recently, it added near-synchronous DR, a blend of synchronous and asynchronous replication. PX-Migrate handles multicloud and multicluster app migrations, as well as snapshots and application-consistent snapshots to the cloud.

PX-Secure constitutes the security layer of the Portworx solution, offering cluster-wide (per-volume) encryption, granular container-based or storage-class encryption (available when organizations bring their own key management system), RBAC, authorization and ownership mechanisms, and integration with Active Directory and LDAP through OIDC.

The solution is managed via PX-Central, a comprehensive management plane that handles multicluster management, command-line interface (CLI) capabilities, proactive centralized monitoring, and cluster installation and setup functions. Integration with Pure Storage Pure1 allows this platform to consume telemetry data from Portworx and deliver app-centric analytics and, eventually, recommendations. Portworx is also natively integrated into OpenShift, with easy deployment options from its UI and deep integrations exposing Portworx storage information in the OpenShift UI.

From an efficiency perspective, the solution handles compression for all snapshots, but true data reduction is achievable only when Portworx uses an underlying enterprise-grade platform with built-in data efficiency capabilities, such as Pure Storage FlashArray.

Strengths: Is a complete enterprise-grade solution with outstanding data management capabilities, unmatched deployment possibilities, and superior management features. It remains the gold standard in cloud-native Kubernetes storage for the enterprise.

Challenges: Data efficiency capabilities are limited when the solution is not coupled with enterprise shared storage, and they are not being addressed on the roadmap short-term. Ransomware protection is not top-of-the-line.

Rakuten Symphony
Symcloud Storage is Rakuten’s innovative, application-aware, cloud-native Kubernetes solution with enterprise-grade capabilities. The solution can run anywhere, either on-premises (containers, bare metal, and VMs) or on all major public clouds.

The product discovers and pools local disks of any type on Kubernetes cluster nodes but can also pool storage capacity from cloud disks and SAN systems. Symcloud Storage delivers a resilient architecture with strictly consistent replicas across cluster nodes, auto-resync for nodes falling behind, and fast-failover capabilities. The solution enables bare-metal performance, live data rebalancing to avoid I/O bottlenecks, and the use of QoS to throttle IO/s usage. QoS isn’t limited to storage but also extends to CPU, memory, and network resources.

Symcloud Storage shines with its advanced data services. Multiple replication modes are supported, with awareness at the node, rack, data center, and zone levels, providing organizations with sufficient granularity. To satisfy application-level deployment and performance requirements, advanced placement capabilities allow organizations to define fine-grained placement policies using affinity/anti-affinity rules. Its management interface includes an application bundles section that provides rapid deployment capabilities akin to an app store experience while respecting best practices deployment topologies for those applications.

The solution also supports snapshots and application-consistent, incremental forever backups. Replication capabilities can be used for data copy and application cloning, disaster recovery, and application mobility across clouds. Data compression is possible, and object storage is supported through integrations with MinIO.

Symcloud Storage supports per-volume encryption, although customers have to operate their own key management system. Monitoring and observability capabilities have been improved, with additional visualizations in the UI, and the data source has also been opened up for scraping using a third-party monitoring tool.

Although it’s available as a standalone product, Symcloud Storage can be coupled with the company’s Kubernetes management solution for a fully integrated infrastructure stack. This solution is well suited to address edge computing use cases. Symcloud Storage has a proven track record with various telcos, for whom edge deployments related to 5G infrastructure are one of the major use cases for containers. It s also integrated into Google Anthos’ marketplace (and is the only pre-certified solution for Google Anthos on-premises) for easier edge deployments and has OpenShift support. Support for Tanzu is missing, but it’s on the roadmap.

Strengths: Symcloud Storage delivers a comprehensive, feature-rich, enterprise-grade experience with an uncompromising adherence to cloud-native development and deployment principles. Advanced data services and application-awareness capabilities are among the highlights of this product, and its backup solution has recently opened up to support non-Rakuten storage.

Challenges: Further improvements in migration capabilities (including onboarding applications not running on their storage nodes and attaching their storage to non-Rakuten clusters), along with security and data footprint optimization capabilities, would further strengthen its position as a Leader.

Red Hat
While IBM’s storage business largely took over Red Hat’s storage portfolio in October 2022, Red Hat OpenShift Data Foundation (ODF) is still available for purchase (albeit with some restrictions, making it available only as part of OpenShift Platform Plus). ODF has become the primary data plane for IBM Storage Fusion, indicating IBM’s commitment to maintain the ODF product.

ODF is a cloud-native storage solution based on Red Hat Ceph, Rook, and Noobaa. The solution is scalable and resilient, and exclusively works with the Red Hat OpenShift platform.

ODF is versatile and supports block, file, and object storage. It can be deployed on-premises or in the cloud and supports snapshots and clones. For data protection, Red Hat s approach is to enable the ecosystem of third-party data protection vendors through its APIs. Advanced data protection features, including replication and disaster recovery, are available only in the Advanced Edition.

ODF delivers capable performance without compromising on data optimization capabilities: erasure coding, compression, and deduplication are currently supported. Multitenancy capabilities go beyond Kubernetes storage classes and include support for ResourceQuotas and LimitRanges, giving organizations control over resource usage and enabling them to overcome the hurdles of workload consolidation and the adverse impact from noisy neighbors.

The solution is excellent from a security perspective, with support for in-flight and at-rest data encryption (at the physical and volume levels). Key management is also supported by ODF. Monitoring and reporting capabilities are good, with integrations into the OpenShift console giving organizations all the basic performance and health metrics.

Finally, edge deployments are also supported when ODF is deployed with OpenShift Container Platform in compact mode, starting at three nodes for OCP and ODF together.

Strengths: A cloud-native storage solution with enterprise-grade features and an innovative approach to cloud deployments, ODF delivers solid value on multiple capabilities.

Challenges: Current support is limited to Red Hat OpenShift. Advanced data services and the approach to data protection remains a weak area.

SUSE
Longhorn is an open-source, cloud-native storage solution originally developed by Rancher Labs and acquired by SUSE. It was accepted by the Cloud Native Computing Foundation (CNCF) in 2019 and is currently an incubating project. Although it’s sponsored by SUSE, it supports any Kubernetes cluster.

Longhorn provides resilient persistent storage for Kubernetes through a two-layer architecture consisting of a data plane and a control plane, by which Kubernetes itself handles the orchestration. The data plane consists of distributed block storage that aggregates and pools the local disk capacity available on each of the nodes. The control plane, via the Longhorn manager, creates volumes by spinning up Longhorn engine instances on the node the volume is attached to and then creates replicas on the nodes where these should be placed. The outcome is a distributed and resilient storage platform with high-performance characteristics. Although Longhorn prioritizes resiliency, performance is adequate and may see further improvements as a result of roadmap development activities.

This solution handles backups and snapshots using a copy-on-write block storage layer that allows point-in-time recovery. Those backups can be exported to either S3 or NFS for offsite storage. The same technology can be used for disaster recovery and replication use cases with an active-passive cluster topology, making multisite disaster recovery possible. A feature called disaster recovery volumes also enables cross-region asynchronous replication in the cloud, with defined RPOs and reduced RTOs.

The solution offers no particular data footprint optimizations, although backups are compressed and based on changed block tracking. Some techniques are used on secondary storage to either reclaim unused space or apply some degree of de-dupe on backup blocks within a single volume. There are no plans to implement data efficiency capabilities for in-cluster storage because of a focus on high performance and resilience. Organizations are thus expected to leverage application-level data efficiency mechanisms.

On the security side, RBAC is supported through Kubernetes, and integration with Rancher technology enables the use of Active Directory and other enterprise-grade authentication providers. In-flight and at-rest encryption for data volumes are supported. Monitoring and alerting are handled through the standard Prometheus and Grafana integrations.

Organizations can deploy Longhorn as a standalone solution or benefit from the strong integration Longhorn has with Rancher. Notably, Harvester is its all-in-one hyperconverged solution integrating Longhorn s storage capabilities with Rancher’s multicluster management capabilities.

Strengths: Longhorn is an interesting choice for those seeking an open source, CNCF-backed storage solution. In conjunction with Harvester, the solution’s migration capabilities are a great fit for organizations looking to bridge the gap between virtualization and cloud-native architectures.

Challenges: Longhorn’s feature list is limited, and some core capabilities are missing, such as support for large volumes (over 2TB) and data footprint optimization. Performance isn’t as good as some competitors, but this is expected to be addressed in 2023 and beyond.

6. Analyst’s Take
The market for persistent Kubernetes storage is moving and innovating quickly, but so are its customers, who are demanding more mature enterprise-grade solutions with each passing year. That means requirements are shifting and becoming stricter each year. This market dynamic is beneficial to customers looking for a Kubernetes-native persistent storage solution, but choosing the right solution in this ever-changing market is paramount and challenging as each vendor is focusing on a different set of priorities.

In this space, we see 2 groups of competitors, roughly divided between those that see persistent storage as their unique differentiation in the market and so are building a product portfolio around it (including various Kubernetes cluster-management solutions), and those for whom storage is just one feature in a larger platform play, usually based on Kubernetes-based developer platforms like Tanzu and OpenShift.

It’s in the first group we see the most complete feature sets, with vendors uniquely positioning themselves against their competition. Discovering which vendor’s positioning best matches your requirements is a prerequisite for long-term success, whether that involves performance, scalability, advanced data service capabilities (like replication or de-dupe), specific deployment models (for edge and other use cases), or developer experience and self-service capabilities.

Similarly, the market has evolved and matured beyond proof of concept and early production environments, and it now has firm security and other enterprise-grade requirements. However, not all vendors have caught up with these demands, and some lack basic security capabilities or even basic data services like snapshots.

It’s worth the effort to investigate a vendor’s support beyond just storage as much innovation is happening in the interface between storage and Kubernetes cluster management, including emerging deployment models for highly integrated turnkey solutions for edge and bare metal.