Cloud-Native, Resilient Edge Cloud Storage Open for Business From Seagate

By Tom Prohofsky, senior engineering lead, office of CTO, Seagate Technology plc

Datasets are the raw ingredients of knowledge. The cloud-native revolution is fueled by an unprecedented rate of developing methods and applications to transform data into information and knowledge.

Seagate‘s internal deployment of smart manufacturing offers just one example.

The amount of potential knowledge is staggering when we consider the diverse types and enormous amounts of data can be gathered via a plethora of sensors and IoT devices. Only the limits set by the available physical storage systems may impede the rate of gaining knowledge.

As businesses attempt to expand data gathering, figuring out where to put this data and managing the costs to get it there can inhibit these gains. As they realize that much of today’s data starts at the edges of the cloud, organizations are looking for solutions to store data at or near the edge to reduce complexity. Scalability and reliability must be central to these solutions.

As a member of The Cloud Native Computing Foundation (CNCF), which builds sustainable ecosystems for cloud-native software, Seagate thinks innovating and enabling cloud-native technologies is critical. Cloud-native computing empowers organizations to build and run scalable, reliable applications with an open-source software stack in public, private, and hybrid clouds.

To be reliable means to be resilient in the face of both normal and challenging situations. For edge cloud storage, normal situations include power spikes, hardware defects, software updates and network loss to name a few. The ultimate goal: applications and services should run effectively despite these events, with little to no effect.

Complete cloud-native edge solutions should offer features that ensure resiliency – both within and between regions of edge deployments.

Storing large amounts of data requires large amounts of HDDs and enclosures connected to servers or to a LAN within a region. When a drive fails, recovery should be resilient and rapid; the time required to heal to regain optimal resiliency should be as short as possible. To rebuild a duplicate of data on a large 18TB drive can take days, when standard RAID methods are involved. Seagate ADAPT data protection technology hardens data by spreading the work across all the drives in the enclosure. This reduces the rebuild time to a few hours, without burdening any network or CPU in the deployment.

Observing the increasing natural disasters of the past several years, it’s clear that a solution to ensure geographical resiliency is also necessary. Geographical resiliency would mean maintaining one or more copies of data in multiple data centers across multiple regions. While mass-capacity mobile storage and data transfer movements can be serviced by the company‘s Lyve Mobile products and services, real-time failover requires a distributed system or synchronous replication.

The company’s CORTX intelligent object storage software provides geographic resiliency for vast amounts of unstructured data objects. For Kubernetes persistent storage, file- and block-level volumes offer agility and performance. Meanwhile, LINBIT’s LINSTOR block storage management for containers provides an open-source storage-defined service platform that works hand-in-glove with Kubernetes Container Storage Interface for orchestrated micro service data placement and scheduling. Moving the container to the edge offers lower TCO than moving data to the core.

The combination of Exos storage platforms and LINSTOR synchronous replication creates a feature-rich, enterprise, resilient solution, open to connect with any other public or private cloud solutions. These platforms’ commitment to open source provides operational agility while providing visibility into all layers of the solution.

Click to enlarge

While the company’s Exos storage can, of course, be used to present block devices to any storage software stack, Seagate has developed an Exos back-end driver specifically for LINSTOR – enabling fine-grain volume control of the Seagate ADAPT hardware RAID engine. With this driver solution in place, hardware-based thin provisioning, SSD/HDD auto tiering and snapshots now are orchestrated by Kubernetes with geo-replication.

Also new to LINSTOR with this solution is mult-host aware resource management. A single Exos system with over 1.5PB of storage can be directly connected to eight application servers, with LINSTOR connecting Kubernetes Persistent Volume Claim to the fault-tolerant logical volumes. Kubernetes using LINSTOR resource definition composes the pairing of compute node and storage connection for both load balancing and agility.

This cloud-native solution is open for business.