Failure Rates Calculated on 171,919 HDDs

This report was written on May 4, 2021 by Andy Klein, director of compliance, on the blog of online backup company Backblaze, Inc.

Backblaze Drive Stats for 1Q21

As of March 31, 2021, Backblaze had 175,443 drives spread across 4 data centers on 2 continents. Of that number, there were 3,187 boot drives and 172,256 data drives. The boot drives consisted of 1,669 HDDs and 1,518 SSDs.

This report will review the quarterly and lifetime failure rates for these data drives, and we’ll compare the failure rates of our HDD and SSD boot drives. Along the way, we’ll share our observations and insights of the data presented, and as always we look forward to your comments below.

1Q21 HDD Failure Rates
At the end of March 2021, the cloud storage company was monitoring 172,256 HDDs used to store data. For this evaluation, it removed from consideration 337 drives which were used for either testing purposes or were drive models for which it did not have at least 60 drives. This leaves with 171,919 HDDs for the 1Q21 quarterly report, as shown below.

Notes and Observations on 1Q21 Stats
The data for all of the drives in data centers, including the 173 drives not included in the list above, is available for download on the HDD Test Data webpagehttps://www.backblaze.com/b2/hard-drive-test-data.html. Uncompressed, there is a little over 68GB of data in 2,900 plus CSV files.

For those of you who reviewed firm’s 2020 annual report, you may notice that the 18TB Seagate drives (model: ST18000NM000J) are not included in this report. This drive model was a customer development unit provided to us for testing purposes and as such should not have been included in the 2020 report tables, as we exclude test drives. This quarter we have correctly categorized that drive model as part our testing cohort and as such that model was not included in this report. Customer development units are mechanically complete, but the firmware is considered non-production and as such the drives are considered test drives.

Four drive models recorded zero failures during 1Q21. Two of these models, the Seagate 6TB and the Toshiba 4TB, have an average age of nearly 6 years, and lifetime annualized failure rates of less than 1% – very impressive over their lifetime. The other 2 models with zero failures, the Toshiba 16TB and the WDC 16TB, have an average age of 4.1 months and 0.4 months, respectively. For these models, having zero failures is a great start and we’ll see how they perform over time. Two other drive models, the Seagate 16TB and the Toshiba 14TB, get honorable mentions as they had only one failure during the quarter. Given both drives are recent additions to our farm, we’ll continue to keep an eye on them as well.

The overall annualized failure rate (AFR) of 0.85% for the quarter is the fourth consecutive quarter where the AFR was below 1%, a trend we are very pleased with, especially as we’ve added over 42,000 new HDDs to te farm and migrated another 23,600 drives over the period.

HDDs vs. SSDs, First Look
Backblaze uses SSD drives in several places, but we currently do not use them for storing customer data – that remains in the realm of HDDs. But one place we have both HDDs and SSDs is as boot drives for storage servers. In our case, describing these drives as boot drives is a misnomer as this cohort is also used to store log files for system access, diagnostics, and more. In other words, these boot drives are regularly reading, writing, and deleting files in addition to their named function of booting a server at start-up.

A little over two years ago, the firm started using SSDs as boot drives. It was about that time we could start getting SSDs that were 200GB or so for less than $50 each, which was our price point for the 500GB HDDs we were buying.

What we have are 2 groups of drives, one HDDs and the other SSDs, which have performed the same functions in the same environment over time. The table below compares the failure rates in aggregate for 1Q21 of HDD and SSD boot drives.

Why didn’t we break these out by model? None of the models by themselves had enough drive days to be statistically relevant. In aggregate, the number of drive days is still on the lower side, but the obvious difference in the AFR between the HDD and SSD boot drives is eye-opening.

If we look at the lifetime results for the HDD and SSD boot drives, the difference in AFR is less, but still significant.

SSD Reporting Moving Forward
One obvious takeaway from these tables is that SSDs fail less often than HDDs, at least in this use case. But that ignores one important factor, drive age. If we focus on the age of each of the cohorts, there are potential cracks in our “SSD drives are better” supposition.
• The average age of the SSDs is 12.7 months, and the average age of the HDDs is 49.6 months.
• The oldest SSDs are about 30 months old and the youngest HDDs are 24 months old.
• The oldest HDDs are nearly 96 months old-8 years old.

Basically, the timelines for the age of the SSDs and HDDs don’t overlap very much, and, in general, drive failure rates typically increase as drive population ages. These 2 considerations make the conclusion that SSDs fail less often than HDDs not as clear cut as it first seems. Over the coming months, we’ll dig into the data and align the SSD and HDD timelines to examine the HDDs in their early years of use and we’ll publish those results. This will give us better insight into the failure rate profile over time for the HDDs.

In addition to the boot drives, Backblaze also utilize SSDs for different use cases, for example on restore servers and so on. Over time, its goal is to instrument these drives as well without impacting performance, so it can build a library of SSD failure rates by use case.

Lifetime HDD Stats
The chart below shows the lifetime AFR of all of the HDDs in production as of March 31, 2021.

Notes and Observations on Lifetime Stats
The lifetime AFR for all of the drives in the farm continues to decrease. The 1.49% AFR is the lowest recorded value since we started back in 2013. The drive population spans drive models from 4TB to 16TB and varies in average age from 2 weeks (WDC 16TB) to nearly 6 years (Seagate 6TB).
Best performing drive models by drive size are listed in the table below.

The average age and drive days for the 14TB and 16TB drives are not sufficient enough to come to any conclusions, but the remaining drives have proven their worth over time in our environment.
Another way to look at this data is by grouping all the drives of a given size: 4TB, 6TB, etc. We’ve done that in the table below.

HDD Stats Data
The complete data set used to create the information used in this review is available on HDD Test Data page. You can download and use this data for free for your own purpose. If you just want the summarized data used to create the tables and charts in this blog post download the Zip file containing the CSV files for each chart.

Read also:
Top News: Backblaze Stats on 150,757 HDDs
Zero failure for Seagate 16TB/18TB, 4TB HGST/WDC and Toshiba models up to now
October 26 2020 | Press Release
Top News: Backblaze Tested 139,867 HDDs in 2Q20
Lowest historical annualized failure rate of 0.81% vs. 1.07% in 1Q20, no failure on Toshiba 4TB drives since 4Q18
August 20, 2020 | Press Release
Top News: HDD Failure Rates Based on 129,764 HDDs Running at Backblaze
1.07%, lowest percentage for any quarter since 2013
May 15, 2020 | Press Release