Addressing Challenges of Public Cloud Storage with Lightbits Cloud Data Platform on AWS

By Gabriel Chapman, senior solutions engineer, Lightbits Labs Ltd.

SAN have long been a popular choice for customers who are looking for high performance, predictability, scale, and durability for their mission-critical data on-premises while at the same time allowing customers to scale compute and storage independently of each other.

Many organizations have made significant investments in SAN technologies over the years, and have architectured robust infrastructure platforms to address their growing data needs. At the same time, these organizations have made significant investments in the public cloud and are moving massive amounts of data out of their own data centers to support the larger shift towards cloud-native applications and operational models.

Subsequently, public cloud storage offerings have not always afforded customers the flexibility of compute instance types being available for existing block storage offerings and the robust data services that they have come to expect with their on-prem storage systems and this means there are gaps that need to be addressed. Oftentimes public cloud storage offerings look more like traditional DAS and incorporate all of the difficulties associated with that operational model. Lightbits Labs believes that it have developed a solution that can afford customers all of the benefits of a traditional SAN that customers have come to rely on, while at the same time, achieving the benefits of what public-cloud technologies offer with better TCO dynamics, simplicity, scalability, performance, and flexibility. This approach also provides a level of predictability that addresses common challenges with cloud-based storage offerings that cannot offer predictable cost, performance, latency, and scale that they had come to expect with their on-prem systems.

Comparing traditional on-prem SAN to public cloud storage
There are several benefits to taking a cloud SAN approach:

Scalability: Traditionally, adding storage in the data center was a time-consuming process that involved not only identifying the proper storage solution, but also the time and effort required to identify, procure, and implement additional storage resources. Adding more storage capacity was a time-consuming process, and oftentimes, organizations were forced to buy more capacity than needed due to the unpredictable nature of storage growth. The public cloud is known for its ability to scale, and the basic storage options that customers can leverage all do this exceptionally well compared to how we have had to scale storage resources on-prem. Still, there are areas where the current public cloud storage offerings are overly prescriptive in how they can be leveraged, this is especially true for higher-performance workloads such as relational and non-relational databases. Additionally, the lack of options currently available does not allow customers the same level of choice for all of their storage needs and forces them to utilize higher-cost cloud storage types which are not always available, and that limits the benefits of a common shared storage solution that can address any compute instance type.

Performance: SAN performance for on-prem customers has been an evolving journey over the last 2 decades, and the adoption of flash-based solutions has become the de facto means to address increasing storage performance needs. Still, performance for on-prem workloads comes at a premium, and it also forces organizations to lock in legacy connectivity options such as FC, iSCSI, or IB. These all contribute to the complexity involved with on-prem solutions and is a critical drivers for customers moving to public cloud. SAN in the cloud looks to streamline connectivity over the emerging NVMe/TCP protocol which affords high performance with low latency and utilizes existing networking infrastructure. It also can address challenges with latency-sensitive workloads and provide consistent performance as workloads scale. Furthermore, the wide number of cloud compute instances available for customers to utilize as storage targets allow customers to more properly size for higher performance without the need to disruptively re-architect storage solutions as they would on-prem. IO/s and bandwidth tend to be limited with existing cloud block storage offerings, or are priced in such a way as to be difficult to justify from a budgetary perspective. With the emergence of NVMe over TCP as a new standard for storage connectivity, we can see the result of high-performance connectivity, with very low latency and higher IO/s and bandwidth than can be achieved with iSCSI, NAS, and even DAS storage offerings that currently exist.

Simplicity: storage provisioning for on-prem customers has been met with multiple levels of complexity, most usually due to storage transport protocol choices and the need for dedicated storage networks to be designed and deployed. For years we have built complex storage networks dedicated to servicing data, and have utilized a number of protocols to service data. FC, iSCSI, Infiniband, NAS, and DAS – are all options that customers have used to connect pooled and unpooled data to the host systems that have storage needs. Each of these protocols has its own inherent pros and cons, and with public cloud, the options become less complex and are simplified with choices often being iSCSI, DAS, and NAS. Still one of the major benefits of the public cloud is the rapid ability to provision resources and service data across IP networks. SAN in the cloud should be provisionable within minutes and must come with robust data services that customers have come to rely on such as snapshots, thin provisioning, compression, clones, and auto-healing capabilities.

Lightbits data platform for any cloud:
The company hase a solution for customers who want all of the benefits of on-prem SAN solutions that they have used for years but in the public cloud. Today, with the company’s Cloud Data Platform available in AWS and Azure, customers can scale either up or down based on their application workloads needs. Starting small and scaling up to petaytes is easy and efficient. In addition, it offers high-performance applications that require consistent low latency and high performance and have the ability to provision a fully functional cloud-based SAN in minutes.

With Lightbits Cloud Data Platform in AWS customers can realize following benefits:

• Lightbits on AWS brings enterprise data features to the cloud to allow you to efficiently and economically deploy and manage block storage for critical database and analytic applications with 100% cost, performance, and scalability predictability
• Unified API and CLI interface for all your high-performance block storage on-premise and on public cloud.
• Pay for the storage you use and not the storage you provision.
• Offer a multi-AZ feature-rich block storage system for AWS that supports thin provisioning, no-cost snapshots and clones, auto-scaling, and compression. Features that allow you to deploy the right amount of storage without waste.
• Simple deployment options that allow customers to create an efficient and economical storage cluster in minutes.
• Address the most demanding applications on any size AWS instance without worrying about running out of IO/s or bandwidth capacity.

The public cloud block storage market has been reportedly valued at roughly 70B and is forecast to reach 376B by 2029. As more customers evaluate their options for high-performance block storage use cases, they will need to look at the benefits of SAN in the public cloud. To quickly test-drive Lightbits on AWS go the AWS Marketplace.

Resources:
Lightbits on AWS
Lightbits on the AWS Marketplace
Video: Lightbits on AWS
Video : How to Deploy a Lightbits Cluster on AWS Marketplace