From Dell, Managing AI Storage Requirements

By Louie Correa, CTO, unstructured data solutions team, Dell global channel, Dell Technologies, Inc.

AI has become an enabler of significant competitive advantage and a way to disrupt markets. A recent Accenture survey revealed that AI achievers – those companies that advance AI maturity enough to achieve superior growth and business transformation – attribute nearly 30% of their total revenue to AI and outperform in areas that include customer experience and sustainability.

For organizations across all industries, the opportunities that AI brings are impactful in substantive ways: augmenting human activities in medicine to provide easier detection and better outcomes; mitigating financial risk by detecting fraud and security threats; improving industrial quality control with automation; and innovating everything from vehicles to our food system are just a few ways that AI is transforming our world. Whether you’re a C-level executive or a data scientist, AI is acknowledged as a business game changer.

Driven by greater access to, and the availability of, data, combined with high performance computing systems augmented with powerful GPUs, AI has become an organizational priority, and mainstream technology. In fact, a Harris Poll found that 55 percent of companies reported accelerating their AI strategy due to Covid, and 67% expect to accelerate their AI strategy moving forward.  

But just because AI adoption is increasing doesn’t mean it isn’t without its challenges. Many organizations’ AI initiatives fail, not only due to lack of proper planning, but also due to an inadequate IT infrastructure to move beyond a POC in a cost effective and scalable way. By one estimate, around 80% of AI projects never make it into production. Why?

Storage: Closing gap between data science and business solution
The gap between the data science solution and the business solution comes down to the lack of an integrated and optimized IT infrastructure. It needs to be built correctly from the start, and also keep data in place as POC moves to production, allowing for both the management of data quality and the expansion to various cloud services to enhance data science productivity. 

The challenges with data storage and management, in particular, require planning for data growth so you can extract the value of data as you move forward, especially as you begin more advanced use cases such as DL and Neural Networks, which require more compute and storage power, performance, and scale.

The requirements for AI drive the demand for higher processing power and throughput, and ML only increases these requirements. When your infrastructure doesn’t allow you to scale compute and storage independently, the problems start.

• Time to market is impacted when data is stored on smaller systems that don’t scale out. With limited space you must move data as part of the normalization process, and smaller systems require data to be separated and deleted, which affects the accuracy of models. A storage solution that can scale and eliminate the need to limit data sets will enable you to develop better models.

• Bottlenecks are created by data-intensive workloads when storage can’t scale, which limits performance and can lead to expensive GPUs sitting idle. This results from disparate storage systems that are stressed by the copying and moving of data, and, in turn, are unable to provide the performance needed for GPUs and technology advances. Data ingestion bottlenecks that are created by storage limitations hinder innovation and model accuracy while also impacting data science team productivity.

• Flexibility to leverage multiple protocol support (including NFS, SMB, HTTP, FTP, HDFS, and S3) for multiple use cases is required to ensure that you can meet the needs of different systems, not just the GPUs for the overall workload and not just a single AI machine learning type of environment.

PowerScale family

PowerScale storage – Powering your AI journey
With superior capabilities for low latency, high throughput and massively parallel I/O, Dell Technologies PowerScale is the storage complement to GPU-accelerated compute for AI workloads. It enables to effectively compress the time needed for training and testing analytical models for multi-petabyte data sets. And, as an added benefit, PowerScale All-Flash storage eliminates the I/O bottleneck with up to 18x more bandwidth and can be added to an existing Isilon cluster to accelerate and unlock the value of massive amounts of unstructured data – key to developing accurate models and successful AI initiatives.

Power, flexibility, scalability, and enterprise grade features that are standard on PowerScale platform help you meet challenges of AI:

• Accelerate innovation with up to 2.7x faster performance to accelerate model training cycles.

• Eliminate AI I/O bottlenecks with enterprise grade features, high performance, concurrency, and scalability that delivers faster training and validation of AI models, higher model accuracy, improvements in data science productivity, and maximization of ROI for compute investments.

• Increase model accuracy with deeper higher resolution data sets with up to 119PB of effective storage capacity in a single cluster.

• Take advantage of flexible deployments and cyber resiliency with bundles that allow you to start small and independently scale-out compute and storage for large scale deployments with data protection and security options.

• Improve data science productivity with flexible in-place analytics and pre-validated solutions for faster, low-risk deployments. 

• Validated Designs for AI are built on technologies including Nvidia GPU Acceleration and reference architectures with Nvidia DGX Systems.

The company delivers end-to-end AI solutions, whether in POC or full production, with the PowerScale platform providing the IT architecture building blocks. And tying it all together is the PowerScale OneFS OS, which enables all nodes to seamlessly operate in the same OneFS powered cluster with the enterprise grade features of performance management, data management, security and data protection – all of which can be delivered at the edge, core, and cloud.

Resource:
Dell analytics solutions with PowerScale and Dell Technologies AI bundles to get started today