HPE Accelerates Enterprise AI Adoption with GreenLake for File Storage

By David Yu, senior manager product marketing, HPE Storage

Hewlett Packard Enterprise Development LP (HPE) expands GreenLake for File Storage capabilities with new high-density, all-flash options specifically designed to power large-scale enterprise AI and data lake workloads.

Compared to the currently shipping release of GreenLake for File Storage, our new options provide 4x capacity and 2x the system performance per rack unit. These improvements allow us to scale throughput at AI scale by 2x and reduce power consumption by 50%.⁽¹⁾ Via a new controller with double the processing power and storage units with a 1U 1.3560PB all-NVMe JBOF, customers can accelerate workloads, slash costs, and substantially reduce the data center footprint and power consumption. ⁽¹⁾

With these enhancements, the company takes another step toward enabling customers to achieve enterprise performance, simplicity, and enhanced efficiency – all at AI and data lake scale. We’re helping organizations to better realize the power of AI and unlock more value from their data; we’re empowering their data scientists, line-of-business application owners, and IT staff with increased productivity; and we’re providing customers the means to achieve high sustainability.

Promises made – and fulfilled
In our previous product release announcement, we said these new enhancements were coming, and now they’re here.

With our latest high-density storage shelf, GreenLake for File Storage has increased the capacity density of our high-end offering by 7x over what was released mid-2023.⁽²⁾ Furthermore, and also as promised, it now delivers up to 2.3x the capacity density of Dell PowerScale and Pure Storage FlashBlade.⁽³⁾ It’s with improvements like these that HPE continues to lead the industry.

Raising bar to achieve AI scale
The company’s announcement extends our capabilities higher in the AI and data lake scale performance and capacity spectrum – demonstrating our ongoing commitment and trajectory to stay ahead of our customers’ requirements to preserve and extend their investment in the industry’s most comprehensive scale-out file data solution for demanding AI and data lake workloads.

Click to enlarge

Let’s unpack the notion of industry leadership at AI scale. As a business moving into or expanding AI initiatives, you want your IT to deliver enterprise performance at AI scale that starts with efficient, performant capacity for data aggregation and preparation and extends to model training, tuning, and inferencing to span all the stages of AI, accelerating even the most data-intensive applications. From GenAI and Large Language Models (LLMs) to myriad other AI workloads, GreenLake for File Storage not only delivers that enterprise performance, but also simplicity and enhanced efficiency, all at AI scale. Consider what that means in practice:

Powering enterprise performance at AI scale
Performance at scale limitations can be an issue for AI storage – particularly legacy scale-out NAS solutions with shared-nothing architectures. These solutions can scale capacity, but they can’t scale performance linearly to match that capacity.

GreenLake for File Storage accelerates your most data-intensive applications with enterprise performance at AI scale. This is performance that spans all the stages of AI – from data aggregation, data preparation, and training and tuning to inferencing. And it’s not just performance that reaches a peak at some point in time for a small data set. Instead, it’s fast, sustained performance that spans the entire scale of your data for the most demanding, data-intensive AI applications, including GenAI and Large Language Models (LLMs).

Enterprise performance at AI scale helps you unlock more value from all your aggregated data – delivering faster time to insight and giving you a real-world competitive advantage.

How do we achieve this? GreenLake for File Storage has a disaggregated, shared-everything, highly resilient modular architecture that allows you to scale performance and capacity independently – and it’s designed for exabyte scale. With all-NVMe speed for fast, predictable performance and no front-end caching, data movement between media, or tiered data pipelines, you can supercharge your most data-intensive AI applications.

Harnessing simplicity at AI scale
There’s another problem with legacy file storage for AI: it can require specialized domain expertise and tedious manual operations to manage, maintain, and scale, and that can keep IT and data scientists tied up in routine day-to-day tasks rather than focused on faster time to value for new AI models.

GreenLake for File Storage provides an intuitive cloud experience with radically simple file data management that reduces operational and management overhead for your IT staff and increases productivity for your data scientists and line-of-business application owners. Powered by the HPE GreenLake platform, you get streamlined deployment, simple setup, easy file share creation, fast job completion, and unified storage management with a single cloud console accessible from anywhere, on any device. You also get automated, nondisruptive upgrades. In addition, via the GreenLake platform, you can modernize your data management with a suite of over 50 cloud data services and a unified cloud operating experience.

All of this simplicity and automation has real-world effects: Your data scientists and line-of-business application owners are no longer burdened by cumbersome, legacy file systems that require technical expertise and intricate setup processes just to run. A single global namespace enhances collaboration for faster time to discovery and innovation. And highly compensated subject-matter experts no longer need to expend valuable time and effort performing IT tasks but can instead focus on their key initiatives.

Enhancing efficiency at AI scale
Data-intensive AI models have an insatiable appetite for storage capacity, rack space, and power consumption. But legacy NAS solutions with shared-nothing architectures are unable to efficiently scale out to keep pace with the capacity density, cost/TB, and power efficiency demands of AI workloads.

GreenLake for File Storage, on the other hand, can slash your AI storage costs with 4x the capacity/RU density and half the power consumption – as well as further lower your carbon footprint with data reduction, non-disruptive upgrades, and an AI storage as-a-service consumption model that helps eliminate overprovisioning. You can scale performance and capacity independently for higher efficiency at lower cost, and you can maximize GPU utilization – and therefore GPU ROI – with enterprise performance at AI scale.

Digging a little deeper, GreenLake for File Storage provides no-overhead snapshots and native replication, superior flash efficiency, and improved data reduction via the Similarity algorithm, which, unlike compression and de-dupe, reduces data with both a global and fine-grained approach. Savings are 2:1 for life sciences data; 3:1 for pre-reduced backups, pre-compressed log files, and HPC and animation data; and 8:1 for uncompressed time-series data.⁽⁴⁾

With support for optimized GPU utilization through InfiniBand, Nvidia GPUDirect, and RDMA, GreenLake for File Storage accelerates AI workloads by boosting performance for model training and tuning via faster checkpointing. Front-end host IB connectivity to networks, including the Nvidia Quantum-2 InfiniBand Platform, provides flexibility. Customers can scale to 720PB of effective capacity (with 3:1 data reduction) for large enterprise-scale AI file data.

Making AI real in your enterprise
To power your AI initiatives and compete in today’s marketplace, you need an AI-ready file storage solution that can deliver enterprise performance, simplicity, and enhanced efficiency – all at AI scale. Truly accelerating your AI workloads requires all 3 measures of AI scalability, as each one is essential for AI success, and GreenLake for File Storage is designed to deliver across each of them. It’s a single, flexible solution that spans not just all the stages of AI, but the full scale of your data. HPE GreenLake for File Storage will help you achieve success wherever you are in your journey to tap the power of AI and make it a reality in your business.

⁽¹⁾These improvements are based on comparisons to the most recent shipping product featuring the 2-node, 2U controller boxes and 550TB 2U storage shelves using 30.72TB NVMe SSDs.
⁽²⁾The 7x improvement is compared to the shipping product from the initial release featuring 307TB 2U storage shelves using 15.36TB NVMe SSDs and delivering aggregate read throughput of up to 200GB/s.
⁽³⁾Comparisons are based on raw capacity for capacity density with Dell PowerScale F900 and Pure Storage FlashBlade. 
⁽⁴⁾HPE Storage Substantiation