NetApp Shipping Dynamic Disk Pools for Big Data

Organizations across a range of industries are dealing with the realities that big data presents. Machine- and user-generated data growth is exploding in size and expected to increase at a rate of 40% year over year (McKinsey & Company, Big data: The next frontier for innovation, competition, and productivity, May 2011.)

Because organizations must be able to turn this data into valuable information to drive smarter business decisions, they require a storage solution that delivers continuous data availability for performance levels and simplified management.

To address these customer requirements, NetApp, Inc. is now shipping Dynamic Disk Pools, a data protection technology that improves overall system performance and is available on big data solutions built on the NetApp E-Series platform.

Additionally, NetApp also announced that Whamcloud’s Chroma Enterprise software is available with the NetApp HPC Solution for Lustre to help customers reduce management complexities.

Continuous Availability, Improves Performance
Dynamic Disk Pools is a technology that administers dynamic reconstructions of failed drives, enabling customers to maintain data availability by restoring a system to optimal performance up to eight times faster than with traditional RAID architectures. They also decrease the performance impact of a failed drive by up to 60%. This allows customers to maintain the required performance levels during drive failures, which helps lower overall storage system costs and enables them to quickly analyze data to speed time to results.

"Technologies like NetApp Dynamic Disk Pools would help the Oak Ridge Leadership Computing Facility deliver consistent, reliable performance even in the face of disk failures," said Dave Dillow, advanced systems architect for Oak Ridge National Laboratory, the largest science and energy national laboratory in the U.S. Department of Energy system. "At the scale and component counts of our computing systems, gracefully handling these failures is fundamental to our mission goals. Our early testing of NetApp’s technology has been very positive, and we look forward to seeing it in the market."

Whamcloud Partnership Simplifies Lustre Management
To help customers simplify overall management of their Lustre environments and deal with the complexities related to deployment and management, NetApp has partnered with Whamcloud to incorporate its Lustre Software Manager powered by Chroma as part of the NetApp HPC Solution for Lustre.

The central management system provides customers with a unified view of their Lustre storage environment. In addition to simplifying overall management, it also enables customers to simplify several tasks that are often complex and time consuming such as installation, configuration, maintenance, monitoring, and fault diagnosis. This allows customers to maximize the performance of their storage environment and reduce the costs associated with managing HPC storage.

"High-performance computing applications are only getting larger and more complex, and, as a result, customers are looking for ways to simplify their environments," said Brent Gorda, CEO, Whamcloud. "Combining Chroma with the NetApp solution provides customers with a truly industry-leading storage solution optimized for Lustre environments and delivers unique management capabilities that offer new levels of simplicity. Now customers can spend less time managing their Lustre environment and more time analyzing data to achieve success."

The enhanced NetApp HPC Solution for Lustre bolsters NetApp’s portfolio of big data solutions on its E-Series platform. Big data solutions are built on E-Series and Data ONTAP platforms to process, analyze, manage, and access data at scale to meet customers’ analytics, bandwidth, and content needs across industries, including oil and gas, financial services, media and entertainment, high-performance computing, and public sector. Customers can spark innovation, make better decisions, and drive business outcomes at the speed of today’s business.

Separately today, NetApp and Hortonworks entered a strategic partnership to develop and pretest joint Apache Hadoop solutions using Hortonworks Data Platform to help customers simplify access to their data and gain deeper business insight to make the right decisions faster.

NetApp also unveiled the Open Solution for Hadoop Rack to provide customers with a ready-to-deploy enterprise Hadoop solution.

Addison Snell, CEO, Intersect360 Inc., said: "Our research into big data applications has shown not only the rapid growth in storage requirements, but more specifically a need for vendors who can balance performance requirements, reliability, and simplicity. Based on our own studies, three-quarters of respondents spanning commercial, government, and academia all plan to invest in new technologies for big data with storage solutions optimized for their environments at the top of the list."

Rich Clifton, senior vice president, Solutions and Integrations Group, NetApp, said: "The information and data that surround us are continuing to grow at incredible rates. Customers across various vertical markets have unique storage requirements to help them leverage this data to drive successful business outcomes. By providing a wide range of big data-focused solutions across our portfolio, including those built on the E-Series platform, NetApp is helping customers more effectively and efficiently process, analyze, manage, and access their data. Today’s announcement is further proof of NetApp’s commitment to continued innovation to provide customers with the right solutions and technologies to address their specific big data needs."

Availability
Dynamic Disk Pools technology is shipping on all big data solutions based on E-Series, which are available from sales and channel partners. The enhanced NetApp HPC Solution for Lustre is available from sales and channel partners.