Case Study From Toshiba: Can Storage Ever Be Installed and Forgotten?

By Rainer W. Kaese, senior manager business development, storage products division, Toshiba Electronics Europe GmbH

Even the most stable and reliable storage systems still require some degree of maintenance. Many may believe that firmware updates, software updates and replacement of failed storage components can never be avoided. Really? This does not have to be so, here is a storage system example that has been running for four years without any need for maintenance or unexpected downtime.

Back in 2017, Toshiba installed a ZFS storage system supporting storage infrastructure needs of the ‘Technology Experience Lab’ at NTT Global Data Centers. Since then, the storage system has proven its reliability: except for a scheduled 10 minutes of downtime to install some additional ZFS features, the system runs steadily without component failures in any of the 60 HDDs, SSD, controllers, power supplies, fans, cables or other components.

The lab provides data center managers and their teams with the room and infrastructure to explore innovative approaches for architecting their servers and storage. It enables businesses to evaluate the efficacy of private or hybrid clouds, distributed architectures, and alternative approaches to deliver IT services in a low-risk environment. In addition, its community of users provides support and ideas through meetups, boot camps, webinars, and hackathons.

In total, 102TB of net storage were desired, requiring 240TB of raw storage, utilising the ZFS-based JovianDSS software of the company‘s partner Open-E. This software had proven to offer HA, no single point of failure, and high-flexibility, providing consistent snapshots and instant restore when required. The hardware to support it would need to be reliable and high-performance to take full advantages of the software, supporting several iSCSI block storage targets ranging in size from 10 to 40TB, plus some shared file folders.

The planning stage was the core for the long-term success of the final storage implementation. Toshiba often undertakes research into server implementation in its laboratories. It has resulted in close relationships with a broad range of component and software suppliers coupled with a deep understanding of what works well. Leveraging this knowledge, the team was able to recommend a hardware architecture that worked with Open-E JovianDSS and had proven itself in other projects.

To form the 102TB of net storage, the team selected Toshiba’s 4 TB, 3.5″ SAS Enterprise Capacity Drives (MG04SCA40EA). With their MTTF (Mean Time To Failure) of 1,400,000 hours and a non-recoverable error rate of just ten errors per 1,016 bits read, they were for achieving the reliability required. The 7,200rpm drives achieved a Zpool read performance rating of 12.9x single disk and 8.5xsingle disk for writes. For the ZFS write logs and reach caches, reliable ten DWPD SAS eterprise SSDs with 1.6TB storage from Kioxia Corp. (formerly Toshiba Memory) were selected.

A significant factor in HDD failures is heat, so the HDD enclosure chassis had to be closely reviewed before selection. The team selected AIC, Inc.‘s J4060-01 dual expander, 12GB/s, 60-bay toploader JBOD. The 1400 W 1+1 hot-swap redundant 80+ Platinum power supply ensured electrical efficiency while also fulfilling the reliability requirements. The JBOD also features four 80x38mm hot-swap fans. System testing showed that the coolest and warmest drive temperature difference was just 4°C, confirming that the correct JBOD had been selected. Connectivity to the JBOD was provided by the Microchip Adaptec RAID Adapter ASR-8885 with 8 internal and 8 external ports, run in HBA mode. This model was highlighted as a top choice back in 2017.

Finally, the storage server and JBOD location in the Technology Experience Lab, at the Frankfurt 1 Campus of NTT Global Data Centers, was the last piece of the reliability strategy. Its 60,000m² of space offers network connectivity and its power is derived from 2 separate European power supply networks that reach directly to the server and drive enclosure. Physical security is ensured, while storage reliability is supported by their efficient cooling system.

Until now, the system runs failure-free for four years and the plan is to continue running the system to explore its reliability further – even when the supplier warranty ends for most of the enterprise-grade hardware components in 1 year. Based on field experience with Toshiba enterprise HDDs, the team expects that the storage has many more years of failure-free functionality to offer.

Over the course of its operation, the storage solution has handled a multitude of proof-of-concept activities in the Technology Experience Lab, much to the satisfaction of all involved. “The Technology Experience Lab benefits a lot from this storage solution, and we are truly impressed by its continuous stability. Set up within our highly available data centers, I can’t imagine a more reliable way of running enterprise storage. We’re looking forward to many more years of failure-free operation“, said Dominik Friedel, business development manager, NTT Global Data Centers.

Having reviewed how the storage is used, with countless snapshots and scrubbing, and iSCSI block targets being connected and disconnected, week in, week out, the software has proven to be the optimal choice too.

“I’m proud to see how well this installation works after all these years, but I am also sure that this is not all this setup can do.”

“Excellent performance and reliability are ensured by the ZFS and Linux-based system in combination with first-class hardware, so it can last for decades. And that’s not just a supposition; Open-E has been developing ZFS storage systems for over twenty years and we have similar, long-lasting installations like this in use at customers all over the world“, said Krzysztof Franek, CEO, Open-E.

Perhaps most critical to the success of this project has been the collaboration. Expertise in each field is essential, but is not enough in isolation. Only by bringing the best of what technology has to offer together, and exploring its capabilities and options, it’s possible to successfully fulfill such demanding requirements.

“We are exceptionally proud to have the partner network of Open-E, Kioxia, AIC, NTT and Microchip. Even if your Toshiba HDD drives are incredibly reliable, they still need a reliable system built of many components to deliver real end-user value“, added Kaese.