Using Software-Defined Storage in Hyper-Scale Environments

Storage workloads in modern data centers increasingly require scale-out environments to run demanding enterprise applications.

These hyper-scale architectures can benefit from software-defined storage (SDS) in terms of economic value, flexibility, and operational efficiency, according to FalconStor Software Inc. for 15 years in software-defined storage solutions.

Scale-out workloads such as No-SQL, OLTP, cloud and big data analytics are hungry for performance and capacity to provide appropriate service levels to end users and applications. The architecture of a hyper-scale data center that must grow to meet compute, memory, and storage requirements on-demand often depends on the nature of the applications and business priorities, such as flexible capacity, security, and uptime. Projects typically are driven by the overall cost of ownership, particularly as requirements reach hundreds of petabytes.

“Many modern applications that need these hyper-scale scale-out environments offer built-in resiliency, protect themselves from hardware failures, and can self-heal, which eliminates the need to build in HA at the storage layer,” said Farid Yavari, VP of technology, FalconStor. “That opens the door to using consumer-grade, commodity hardware that can fail without impact on service availability. On the other hand, the relatively smaller footprint of revenue-generating scale-up applications may justify paying a premium for name brand storage with HA and data protection features, because it’s unwise to test radical new technologies in that environment.“

Properly architected SDS platforms enable the use of heterogeneous commodity hardware to drive the lowest possible cost, orchestrate data services such as replication, and create policy-driven, heat-map based tiers, so data is placed on the appropriate storage media. An SDS approach eliminates the reliance on expensive, proprietary hardware and vendor ‘lock-in’.

The two most common models for scale-out hyper-scale storage are either DAS, or a disaggregated model based on various protocols such as iSCSI or NVMe. Some very large custom data center installations at companies that have the right protocol-level engineering staff run on homemade, workload-specific protocols specifically developed to optimize the storage traffic for their custom use cases.

Since the available slots constrain the DAS model in a server, the scale is limited and often quickly outgrown. In the DAS model, independent scaling of compute and storage resources cannot be optimized. Therefore, enterprises start with, or must ultimately move to disaggregated storage models built with commodity hardware.

SDS adds intelligent orchestration and management to the disaggregated data center via an abstraction layer separating heterogeneous storage hardware from applications. SDS results in a more resilient, efficient, and cost-effective infrastructure. The fact that SDS is hardware agnostic allows enterprises to implement new storage technologies in a brown field implementation, eliminating the need for deploying greenfield infrastructure when initially migrating to newer storage models. Using SDS capabilities, the migration from legacy to modern technologies can happen over time, maximizing ROI in an already established storage infrastructure. SDS provides flexibility in data migration, seamless tech refresh cycles, and independent scaling of the storage and server resources.

Even where data protection and HA capabilities aren’t necessary, SDS can provide other valuable features such as actionable predictive analytics, WAN optimization, application-aware snapshots, clones, QoS, deduplication and data compression.

“SDS solutions blend well with hyper-scale infrastructures built to meet growing requirements for storage flexibility, density and performance,” said Yavari. “Falling prices of flash, the introduction of various flavors of storage-class memory, and an increasing appetite for commoditization of the data center infrastructure has helped fuel possibilities in hyper-scale storage. SDS enables deployment of storage technologies with different capabilities, at various cost points, to drive the lowest possible TCO.”

FalconStor’s FreeStor delivers enterprise, software-defined, intelligent data services combined with predictive analytics across any primary or secondary storage hardware; in the cloud or on premise. It helps IT organizations realize more economic value out of existing environments and any future storage investments while maximizing flexibility and operational efficiency.