National Climate Agency Selects Versity Software

A national climate agency chooses Versity Storage Manager (VSM), a large scale archive data management tool from Versity Software, Inc.

VSM helped the national climate agency meet demanding HPC archive performance requirements.

The system currently archives at over 12GB/s to tape, moving 1PB of data per day for one of the most active high performance archive solutions.

Challenge:
The agency runs a variety of numerical and observation simulation experiments to accomplish their mission to improve weather forecasting and to gain greater understanding of the climate on a large temporal scale. Through their simulation endeavors the agency both receives and produces massive amounts of large file data that is used as part of their HPC workflow.

The file data must be available for their a HPC system, but because storing the entire data collection in the parallel file systems is cost prohibitive, they needed a low cost archive solution featuring both capacity and performance. The solution needed to ingest files from their Lustre parallel file system quickly and needed to write 2 copies to tape at a speed of at least 10GB/s under everyday production conditions.

Solution
The national climate agency selected Versity’s VSM product for their archiving system. The software was a good fit for their performance, capacity, and total cost of ownership requirements.

Click to enlarge

Environment Snapshot: 

• 35PB capacity
• 850TB – 1PB write throughput per day
• 12GB/s
• 2 servers (active/passive)
• 6 SAN clients
• 48 LTO-7 drives
• Millions of files

With 48 LTO-7 tape drives on a Spectra TFinity series library and demanding ingest and staging requirements, they required a product that was capable of scale and performance. They selected a company that understands serial devices and how to use them efficiently. Versity met the agency’s performance requirements, and also exceeded the archive to tape speed requirement by over 2GB/s for a total of 12GB/s in performance. The data moves in and out of the archive on a regular basis, moving up to 1PB of data per day.

Versity enabled the national climate agency to deploy the solution on commodity hardware. Their data cache was affordable and met their stringent workload and performance requirements. With Lustre mounted on several VSM nodes, the cache configuration is capable of ingesting data from their Lustre system at over 20GB/s. As files are no longer needed on Lustre they are transferred to the VSM cache where policies are automatically applied and archive copies are sent out to tape. The files are then recalled (staged) when HPC jobs are run that require the files to be moved back into the Lustre system.

Automatic tiering and a single interface for all archive data made archiving faster and easier for the national climate team. The agency appreciates the way cache space is managed automatically, relieving worry that the file system may get too full, freeing IT staff time for other tasks.

About the National Climate Agency:
It provides services in forecasting monsoons and other weather/climate events for its host nation. It also helps with science and technology for exploration and development of ocean resources, and plays an important role for Antarctic/Arctic and Southern Ocean research.