Quantum Myriad Software-Defined All-Flash Storage Platform for Enterprise
Cloud-native file and object storage software ends constraints of legacy hardware-centric systems.
This is a Press Release edited by StorageNewsletter.com on April 4, 2023 at 2:02 pmQuantum Corporation announces Myriad, an all-flash, scale-out file and object storage software platform for the enterprise.
Leveraging advances in application frameworks and design that were not available even a few years ago, Myriad’s cloud-native architecture makes it an easy-to-use solution that overcomes the limitations of hardware-centric designs and enables customers to adapt to future storage needs while reducing the burden on over-extended IT staff.
The amount of data in the enterprise is expected to double over the next 5 years, according to IDC. Yet most enterprises continue to store this data on systems that were designed 20 years ago and cannot take advantage of the latest flash and memory technologies. Even more recent all-flash file and object storage solutions rely on specialized hardware and won’t run natively in the cloud, leading to tradeoffs, compromises, and complexity.
Myriad solves these challenges with an a new shared-nothing architecture designed for the latest flash technologies to deliver consistent low-latency performance at any scale. It introduces inline data services such as de-dupe and compression, snapshots and clones, and metadata tagging to accelerate AI/ML data processing. And it uses familiar and proven cloud technologies, like microservices and Kubernetes, to deliver cloud simplicity wherever deployed. The software operates on standard high-volume flash storage servers so IT teams can quickly adopt the latest hardware and storage infrastructure for future needs. With Myriad, enterprises can replace legacy disk-based storage systems with a software-defined, all-flash platform that provides faster performance, greater scale, and a more sustainable and green solution that is both power and real estate efficient.
“To keep pace with data growth, the industry has ‘thrown hardware’ at the problem,” says Brian Pawlowski, chief development officer. “Integrating feedback from customers, partners and industry analysts on what’s missing from current solutions in market, we took a totally different approach with Myriad, and the result is the architecture I’ve wanted to build for 20 years. Myriad is incredibly simple, incredibly adaptable storage software for an unpredictable future.”
Myriad expands company’s portfolio of solutions for unstructured data and is for emerging high-growth use cases that require more performance and more scale, including AI and ML, modern data lakes, VFX and animation, and other high-bandwidth and high IO/s applications. These use cases are driving growth in the market for scale-out file and object storage, expected to grow at an 12% CAGR and be a $19.1 billion market by 2026*.
Myriad’s key features include:
- A new, shared-nothing software architecture built for the latest flash technologies to deliver consistent, low-latency performance at any scale.
- A modern microservices architecture, orchestrated by Kubernetes to provide a resilient, “always on” architecture and deliver new features and fixes rapidly with less risk.
- Self-healing, self-balancing software for in-service upgrades that automatically rebuilds and repairs data in the background while also rebalancing data as the storage cluster expands, shrinks and changes.
- Automated detection, deployment and configuration of storage nodes within a cluster so a cluster can be scaled, modified or shrunk non-disruptively, without user intervention.
- Automated networking management of the internal RDMA fabric so managing even a large Myriad cluster requires no networking expertise.
- Inline data de-dupe and compression to reduce the cost of flash storage and improve data efficiencies relative to legacy storage platforms.
- Simple data security and ransomware recovery with built-in snapshots, clones, snapshot recovery tools and “rollback” capabilities.
- Inline metadata tagging to accelerate AI/ML data processing, provide real-time data analytics, enable rapid creation of data lakes based on tags, and automate data pipelines and workflows.
- Real-Time monitoring of system health, performance, capacity trending, and more from a secure online portal by connecting to Quantum cloud-based AI Operations software
“To power the next era of digital business, we need an evolution in storage that removes the constraints from yesterday’s data center,” said Dave Pearson, research VP, infrastructure systems, platforms and technologies group, IDC. “For new demanding workloads, low latency and tremendous IO/s are required, along with increased scalability and native cloud interoperability to power the next wave of applications.”
Myriad is available for early access customers and is planned for availability in 3Q23.
*Source: IDC WW File and Object Based Storage Market Update and Forecast, Dec 2022. Excludes Shared Cloud revenues.
Comments
This is a significant news for Quantum and the market.
For Quantum, it marks its official entrance to the generic primary storage landscape, in the unstructured segment with a file and object product, ready to battle vs. Pure Storage, Qumulo, Vast Data and Weka.
For the market it confirms the current trend with flash-based file storage designed with the new media in mind, of course NVMe, its network companion, and the containerization.
And we’re not sure competitors have expected a new offering from Quantum, except ones who saw the arrival of Brian Pawlowski, who spent 3 years at Pure Storage, more than 21 years at NetApp and almost 7 years at Sun Microsystems. Quantum confirmed that the project has started before Pawlowski joined the team, but clearly his impact is visible.
The company is well recognized for its vertical solutions such StorNext in the M&E, a file storage solution for primary storage but for specific usages. It also identified Atavium in the domain and acquired the company to offer ATFS, no longer present on the web site. So clearly the firm recognized the need of a new solution born in the era of flash, NVMe and container which is not the case for StorNext or even Isilon, 2 good products designed more than 2 decades ago.
So what is Myriad?
First, the main idea is to deliver a modern file storage product to address new needs around low latency, high throughput and IO/s for unstructured data based on current technology. It is a scale-out file and object storage platform purely software able to run on-premise and in the cloud on commodity hardware, in other words a good example of a software-defined storage concept. Initially Quantum will provide the hardware, like Qumulo and Vast Data did when they released their product at the beginning. The team also has chosen a microservices model orchestrated by Kubernetes as all services are isolated and containerized for better management, scalability, mobility and scale, promoting a real example of a cloud-native implementation. The other idea is based on the autonomous capability of the cluster to adapt itself to changes.
The product relies on a cacheless model with a layer of balancer nodes, and a storage layer covered by NVMe storage nodes, essentially 1U Linux storage server with 10 NVMe flash TLC SSDs, exposing access protocols outside. The configuration starts with 5 storage nodes and 2 balancer nodes plus one deployment node, outside the data path for a total of 8 nodes so 8U. Every 10 storage nodes, 2 balancer nodes must be added and of course all balancer nodes are connected to all storage nodes. Storage nodes are equipped with 100GbE RDMA NIC like the balancer nodes operating as Linux-based switches. There is no state stored on the front-end nodes and it appears to us that the architecture has some common ideas we found at Vast Data with some others also from Pure Storage. Each storage node owns and controls its internal SSDs without any direct connection to others drives placed in others 1U chassis. This is why Quantum qualifies its architecture as shared-nothing. Again the idea is to offer parallelism at scale with no bottleneck. The cluster is managed via the deployment node and comprehensive GUI.
The product appears to be rich even if all features are not ready yet for the GA coming in a few months but the direction is set. Various access methods are provided, only NFS v4 at the launch, NFS v3 and SMB in 2 quarters and S3 in 4 quarters following the GA, coupled with several data services like inline de-duplication, compression, snapshots, clones, and replication. Indexing and data classification will appear later. NFS is based on Ganesha, SMB on another commercial offering. Developing an efficient SMB is not trivial and some good independent stacks exist on the market and Quantum adopted a reference in the domain like others players in the industry. All traffic enter the cluster via a single IP address using BGP.
The interesting part is the file system layer with multiple instances of them above a transactional key-value stores who store data states and values, we mean metadata and data. This KV resides on top of a data protection layer that implements a flexible dynamic N+M erasure coding service. This EC uses the concept of zones sets, like a group of SSDs partitions transversal across storage nodes, but here a group of zones. At the launch, a +2 protection scheme is offered.
All content is stored as object internally and based on their size, as one object or several chunked entities. The key attribute of this KV resides in its transaction flavor preserving the POSIX requirements.
We found some common ideas with other competitive products mentioned above that serve as a reference as they’re good ideas and some of them represent the state of the art.
With this announcement, Quantum, one of the secondary storage champion, confirms that the data center is going to be full flash and is ready to shake established position …
Subscribe to our free daily newsletter
