Can Storage Ever Be Installed and Forgotten?

The author of this blog is Rainer W. Kaese, senior manager business development, storage products division, Toshiba Electronics Europe GmbH.

Can Storage Ever Be Installed and Forgotten?

Even the most stable and reliable storage systems still require some degree of maintenance. Many may believe that firmware updates, software updates and replacement of failed storage components can never be avoided. Really? This does not have to be so, here is a storage system example that has been running for four years without any need for maintenance or unexpected downtime.

Back in 2017, Toshiba Electronics Europe GmbH installed a ZFS storage system supporting storage infrastructure needs of the Technology Experience Lab at the Frankfurt 1 Campus of NTT Global Data Centers. Since then, the storage system has proven its outstanding reliability: except for a scheduled 10mn of downtime to install some additional ZFS features, the system runs steadily without component failures in any of the 60 HDDs, SSD, controllers, power supplies, fans, cables or other components.

Tech Lab Exploring New Architectures
The Technology Experience Lab provides data center managers and their teams with the room and infrastructure to explore innovative approaches for architecting their servers and storage. It enables businesses to evaluate the efficacy of private or hybrid clouds, distributed architectures, and alternative approaches to deliver IT services in a low-risk environment. In addition, its community of users provides support and ideas through meetups, boot camps, webinars, and hackathons.

In total, 102TB of net storage were desired, requiring 240TB of raw storage, utilising the ZFS-based JovianDSS software of Toshiba’s partner Open-E GmbH. This software had proven to offer high availability, no single point of failure, and high-flexibility, providing consistent snapshots and instant restore when required. The hardware to support it would need to be reliable to take full advantages of the software, supporting several iSCSI block storage targets ranging in size from 10TB to 40TB, plus some shared file folders.

Extensive Planning Was Key Factor
The planning stage was the core for the long-term success of the final storage implementation. Toshiba often undertakes research into server implementation in its laboratories. It has resulted in close relationships with a range of component and software suppliers coupled with an understanding of what works well. Leveraging this knowledge, the team was able to recommend a hardware architecture that worked with Open-E JovianDSS and had proven itself in other projects.

Inside the Technology Experience Lab

Reliability and Performance
To form the 102TB of net storage, the team selected Toshiba’s 4 TB, 3.5″ SAS enterprise capacity drives (MG04SCA40EA). With their MTTF of 1,400,000 hours and a non-recoverable error rate of just 10 errors per 1,016 bits read, the drives have been suitable for achieving the reliability required. The 7,200 rpm drives achieved a Zpool read performance rating of 12.9 x single disk and 8.5 x single disk for writes. For the ZFS write logs and reach caches, reliable 10 DWPD SAS Enterprise SSDs with 1.6TB storage from Kioxia (formerly Toshiba Memory) were selected.

A significant factor in HDD failures is heat, so the HDD enclosure chassis had to be closely reviewed before selection. The team selected AIC’s J4060-01 Dual Expander, 12GB/s, 60-bay toploader JBOD. The 1400W 1+1 hot-swap redundant 80+ Platinum power supply ensured electrical efficiency while also fulfilling the reliability requirements. The JBOD also features four 80×38 mm hot-swap fans. System testing showed that the coolest and warmest drive temperature difference was 4°C, confirming that the correct JBOD had been selected. Connectivity to the JBOD was provided by the Microchip Adaptec RAID Adapter ASR-8885 with 8 internal and 8 external ports, run in HBA mode. This model was highlighted as a top choice back in 2017.

Finally, the storage server and JBOD location in the Technology Experience Lab was the last piece of the reliability strategy. Its 60,000m² of space offers network connectivity and its power is derived from 2 separate European power supply networks that reach directly to the server and drive enclosure. Physical security is ensured, while storage reliability is supported by their highly efficient cooling system.

No Failure in 4 Years
Until now, the system runs failure-free for 4 years and the plan is to continue running the system to explore its reliability further – even when the supplier warranty ends for most of the enterprise-grade hardware components in one year. Based on field experience with Toshiba Enterprise HDDs, the team expects that the storage has many more years of failure-free functionality to offer.

Over the course of its operation, the storage solution has handled a multitude of proof-of-concept activities in the Technology Experience Lab, much to the satisfaction of all involved.

“The Technology Experience Lab benefits a lot from this storage solution, and we are truly impressed by its continuous stability. Set up within our highly available data centers, I can’t imagine a more reliable way of running enterprise storage. We’re looking forward to many more years of failure-free operation,” said Dominik Friedel, business development manager, NTT Global Data Centers.

Success Story of Partners Involved
Having reviewed how the storage is used, with countless snapshots and scrubbing, and iSCSI block targets being connected and disconnected, week in, week out, the software has proven to be the optimal choice too.

“I’m proud to see how well this installation works after all these years, but I am also sure that this is not all this setup can do. Excellent performance and reliability are ensured by the ZFS and Linux-based system in combination with first-class hardware, so it can last for decades. And that’s not just a supposition; Open-E has been developing ZFS data storage systems for over 20 years and we have similar, long-lasting installations like this in use at customers all over the world,” said Krzysztof Franek, CEO, Open-E.

Perhaps most critical to the success of this project has been the collaboration. Expertise in each field is essential, but is not enough in isolation. Only by bringing the best of what technology has to offer together, and exploring its capabilities and options, it’s possible to fulfill such demanding requirements.

“We are exceptionally proud to have the partner network of Open-E, KIOXIA, AIC, NTT and Microchip. Even if your Toshiba Hard Disk Drives are incredibly reliable, they still need a reliable system built of many components to deliver real end-user value,” added Kaese.