DataCore

SSDs to Complement, Not Replace HDDs in Data Centers – Gartner

Within next 5 to 10 years
This is a Press Release edited by StorageNewsletter.com on 2013.04.30

AddThis Social Bookmark Button

Valdis Filks, Joseph Unsworth, John Monroe, Stanley Zaffos, analysts from Gartner, Inc., wrote the following paper, dated April 15, 2013:

Solid-State Drives Will Complement,
Not Replace, Hard-Disk Drives in Data Centers

Given the enormity of future capacity needs, it will be impossible for CIOs and IT managers to completely replace HDDs with SSDs during the next five to 10 years in data centers. Although SSDs will increase hardware performance, application workloads will still require software optimization.

Overview


Impacts

SSDs will become ubiquitous as IT departments will use some SSD capacity for all performance-constrained data center applications.
IT departments will replace fast HDDs with SSDs, which will increase the need for slower, larger HDDs for storing inactive data.
CIOs and IT managers will be under constant pressure from SSD vendors to buy the latest and fastest storage device, even when it is not required.

Recommendations

  • Do not plan to replace all HDD storage within your IT infrastructure with all SSD storage.
  • Continue to holistically monitor systemwide application performance, as bottlenecks solved by SSDs can move to other areas, such as the Ethernet network.
  • Verify that all SSD arrays replacing HDD arrays have the same or better reliability and equivalent high-availability features, such as dynamic nondisruptive software/firmware and hardware upgrades and replacements.
  • Ensure that your choice of system and management software will allow for seamless integration with snapshots, replication and intelligent tiering of data among heterogeneous storage devices.

Analysis
The benefits of SSDs are immediately visible to users as a means to improve application performance by reducing electromechanical HDD latencies. In addition, the ROI of SSD performance benefits is immediately apparent and measurable, especially when the applications using the SSDs are high-profile, revenue-generating services. Installing faster components, such as SSDs, into storage systems and servers helps IT departments avoid outlays for time-consuming projects that require specialist application software and infrastructure performance-tuning skills. However, the cost per GB of enterprise-grade SSDs, compared with enterprise-grade HDDs, will remain at prohibitively high levels for the foreseeable future (the cost per GB for enterprise server SSDs will remain at more than 25 times the cost per GB of enterprise business-critical HDDs). Therefore, it will not be economically feasible for IT departments to simply replace HDDs with SSDs within the next five to 10 years.

There will continue to be enormous differences between the costs and efficiencies of petabyte - quantity production - for example, when looking at enterprise storage on the devices that were destined for use solely in servers and storage systems. In 2012, the HDD industry delivered 66,358PB in 63.4 million business-critical and mission-critical HDDs, whereas the NAND industry delivered only 1,781PB in 5.7 million enterprise-grade server and storage SSDs.

It would cost the NAND industry hundreds of billions of dollars to construct enough new fabrication plants to displace even 20% of the forecast need for enterprise storage, which likely will exceed 500,000PB in 2017. Therefore, within the next five to 10 years, it will be physically impossible to manufacture a sufficient number of SSDs to replace the existing HDD installed base and produce enough to cater for the extra storage growth.

                 Impacts and Recommendations
        for Use of SSDs and HDDs in Data Centers
gartner_fig_1_540
Source: Gartner (April 2013)

SSDs will become ubiquitous as IT departments
will use some SSD capacity
for all performance-constrained data center applications

Most computers and storage devices are required to temporarily or permanently store highly accessed and performance-sensitive data. By adding SSDs to computer systems, the performance of the whole system increases. For some components, such as CPUs, the rate at which they process data increases by an order of magnitude. Overhead processes, such as swapping and paging, are avoided. The result is an increase of CPU utilization or, more accurately, a reduction in CPU task or thread wait time. However, bottlenecks and resource contention can move to other areas, such as the network, an internal system bus or interconnects.

The benefits of running databases and data warehouses directly in-memory, via DRAM and complemented with nonvolatile flash memory in the managed form of SSDs, are compelling when considering the ultrahigh performance and negligible latencies particularly attractive for in-memory DBMSs (IM DBMSs), in-memory data grids (IMDGs), high-performance message infrastructure, in-memory analytics tools, complex-event processing (CEP) platforms and in-memory application servers (IMAS). Gartner expects that these in-memory computing (IMC)-enabling application infrastructure technologies will be mature and poised for traction beginning in 2015, led by several prominent vendors and a flood of smaller vendors.

Growing adoption of IMC application infrastructure technologies for analytics will be used initially as a competitive advantage, enabling IT organizations to provide users with faster, more-detailed analysis of business data. As IMC gains make deeper inroads into IT organizations, the technology will be leveraged increasingly for cost savings. Although memory will be used to run these extremely high-performance applications, HDD technology will often remain the most manageable and cost-effective means to store data indefinitely. For organizations that cannot justify the purchase costs of IMC but also cannot tolerate the slow response of HDD, SSDs are an excellent compromise between high costs and slow performance.

Recommendations:

  • Implement and use SSDs in all systems as long as the business or organizational benefit is worth the investment.
  • Verify that the SSD arrays replacing HDD arrays have the same or better reliability and equivalent high-availability features, such as dynamic nondisruptive software/firmware and hardware upgrades and replacements.
  • Embrace IMC as a means of competitive differentiation, but recognize that HDDs will often remain the most convenient and inexpensive recovery and archival media in the market.
  • Continue to holistically monitor systemwide application performance, as bottlenecks solved by SSDs can cause congestion in other areas, such as storage or Ethernet network.

IT departments will replace fast HDDs with SSDs, which will increase the need for slower, larger HDDs for storing inactive data
The incursion of enterprise-grade SSDs in servers and storage systems will continually erode the need for fast HDDs. This will fuel massive growth in the demand for capacity-optimized, business-critical HDDs as the preferred devices for storing expanding quantities of infrequently accessed data. This SSD incursion does not eliminate HDD technology from data centers; rather, it will slowly displace the highest-performance 15Krpm mission-critical drives and, to some extent, will push into the midrange 10Krpm high-performance HDD segment. This trend is well under way, with virtually all the inefficiently data-striped 15Krpm HDDs being replaced by SSDs. A significant driving factor is the reduced amount of operational costs, such as manual tuning that is required with HDDs and the earlier mentioned quick performance benefits (Use SSDs, Rather Than Disk Striping, to Improve Storage Performance and Cut Costs (9 pages, $495).

In the foreseeable future, HDDs will have an advantage in terms of capacity and price per GB over SSDs. Less obvious is the noteworthy fact that the performance benefits enabled by SSDs and enhanced by software optimization will fuel massive growth in the demand for 7.2Krpm capacity-optimized, business-critical HDDs as the preferred devices for storing expanding masses of infrequently accessed data.

The access patterns, value and usage characteristics of data stored within storage arrays varies widely, and is dependent on business cycles and organizational work processes. Because of this large variability, data stored within storage devices cannot be economically and efficiently stored on the same storage type, tier, format or media. To develop the most-scalable and best-optimized configurations, IT professionals must use storage resource management, file analysis tools or performance- and capacity-planning tools to understand their current data variability, access patterns and total capacity needs, and project what these demands will be in the near and long term.

Recommendations:

  • Calculate the operational expenditure of all storage through a total cost of ownership (TCO) model, which includes power and space savings plus benefits from potential server consolidation.
  • Use HDDs for the majority of an organization's active data with SSD specifically installed and implemented for use with applications and data that require fast access times and high performance.
  • Use IMC technology to support the most-latency-sensitive and high-performance applications.
  • Use SSDs within a tiered storage architecture in concert with HDDs to solve environmental problems, such as power usage, cooling and floor space.
  • Ensure that your choice of system and management software will allow for seamless integration and intelligent tiering of data among disparate devices.

CIOs and IT managers will be under constant pressure
from SSD vendors to buy
the latest and fastest storage device,
even when it is not required

SSD implementation projects are relatively quick and simple to implement, compared with most storage projects. The performance results of these projects are immediately noticeable, measurable and, if chosen correctly, increase production efficiencies, reduce process time and generate capital. Consequently, the temptation to buy and implement more SSDs is high. Therefore, even when the business case does not justify the investment, CIOs and IT managers will be pressured by SSD vendors to buy more SSDs and implement quick-win projects, which enable order of magnitude application performance gains.

The direct purchase cost of the storage media should not be the only consideration. Buyers also must consider other variables that could factor in heavily during a TCO decision. SSDs have the advantage in terms of lower power consumption directly and indirectly via lower power and cooling requirements. However, these advantages allow only minor cost savings (e.g., depending on scale and power rate), because the storage power budget in a data center is roughly only 10% of the whole budget. Often more compelling from a TCO perspective is the potential for server consolidation to offer a more favorable density per rack. Lastly, the improved CPU utilization can lead to server consolidation due to a reduction of performance bottlenecks, which also potentially reduce software licensing costs, providing further justification for SSD purchasing.

Recommendations:

  • Do not use SSDs for all storage requirements, but only for high-value data, which also requires fast access.
  • Review data residing on SSDs and its business value on a continuous basis.
  • Invest in SSDs to reduce and recover costs in related IT infrastructure devices, and free up capital to increase other IT infrastructure investments and improve overall IT agility and costs.
  • Do not plan to replace all HDD storage within your IT infrastructure with SSD storage.

This table compares the projected component average selling price (ASP) per GB for large-scale system integrators, and OEMs for HDDs and SSDs. Note that the costs of more-expensive (enterprise mission-critical HDD and enterprise storage SSD) and less-expensive (enterprise business-critical HDD and enterprise server SSD) devices have been compared separately, to show that even the most aggressively priced enterprise SSDs will remain distant in terms of price parity.

 Comparative Costs per GB of Enterprise HDDs and SSDs
                        (Based on Vendor Revenue)
gartner_tab_1_540
 Source: Gartner (April 2013)

This table compares the average capacities of enterprise HDDs and enterprise SSDs.

                 Comparative Average Capacities
                of Enterprise HDDs and SSDs (GB)
gartner_tab_2_540
Source: Gartner (April 2013)

Although many innovative SSD vendors are using data compression and data deduplication technologies on active data to improve the viability and competitive cost of SSDs, enterprise mission-critical SSDs will remain relatively expensive on a cost-per-GB perspective and small in capacity compared with enterprise storage HDDs. There will continue to be enormous differences between the bare-bones costs and capacities of the two technologies. Enterprise HDD shipments will greatly exceed enterprise SSD shipments through 2017. Therefore, even when the utilization of SSDs is increased by these data reduction technologies and SSDs are measured by performance per GB or cost per IOPS, HDDs will still be extensively used within data centers for the next five to 10 years.

This table shows Gartner's forecasts for enterprise HDDs and enterprise SSDs.

 
Shipments of Enterprise HDDs and SSDs, 2010 to 2017
                         (Thousands of Units)
gartner_tab_3_der_540
 Source: Gartner (April 2013)

IMC Definition
Gartner defines IMC as "a computing style, which assumes that the primary data store for applications (the 'database of record') is the central (or main) memory of the computing environment running these applications."

SSD Is Secondary Storage, Not Memory
IMC is not the same as SSD or HDD storage because different mechanisms are used to store and access data within these devices. If data is to be processed by CPUs, then it has to reside in DRAM (Level 1 or Level 2 CPU cache). Data on SSD or HDD cannot be directly accessed, addressed of processed by CPUs. Due to the decrease in DRAM prices, the data required by high-performance, mission-critical applications is being moved from being permanently stored on HDD and SSD storage to server memory - i.e., DRAM. This causes a knock-effect where the application data moves from HDD to IMC (DRAM). As the data is processed, accessed less or becomes inactive, it is moved back from DRAM to SSD. HDD, if used at all, is only used for bulk storage of nonperformance critical data.

16 665
Datadobi