7.8PB for UK Users of HECToR HPC

A new storage environment providing 7.8PB of storage and an additional 19.5PB of backup capability is to improve long-term data storage for hundreds of UK users of the HECToR (High-End Computing Terascale Resource) supercomputer.

HECToR is hosted by EPCC at the University of Edinburgh and funded by the Engineering and Physical Sciences Research Council (EPSRC), and the Natural Environment Research Council (NERC).
 
The additional storage compliments HECToR’s existing 1PB of disk space. Although tightly integrated with HECToR, the new storage environment is built independently and – because it is designed to out-live HECToR – will be available for use with successive supercomputers.
 
The storage environment was designed and built by data processing, data management and storage provider OCF plc. It uses storage hardware from DataDirect Networks, Inc. (DDN), and archive hardware and file management software from IBM Corp.
 
"We needed a more a data-centric view of high performance computing," says professor Arthur Trew, University of Edinburgh. "Data persists beyond any computer, including HECToR, so we’re prioritising data storage, management and analysis. Doing this enables us to upgrade HECToR and integrate its successor without fear of impacting access to research data. Our expectation is that any future computer must be able to integrate seamlessly with our storage."
 
Scientists currently store complex simulations on site at Edinburgh – file sizes vary from user to user, but each can potentially be gigabytes in size. The passage of data for further interrogation is unique to each researcher and may involve transferring the data to other data repositories off site, moving data to different parts of the country or simply ‘taking it home’ using portable media.   
 
Julian Fielden, OCF MD, says: "There is lots of talk and consensus at the moment that the problem with big data isn’t really the capacity to store it, but how to access, use and find the data and, in doing so, make it into useful information. The collective investment of the research councils is cleverly helping to avoid this problem by making storage independent of the machine that generated it. Combined with good network access and IBM’s parallel file system GPFS, the data becomes easy to locate and use by any researcher irrespective of location."
 
"As we enter the big data era, organisations in every field of endeavour are addressing the World’s most pressing scientific and medical questions – questions that would have been too complex to address just a few years ago," says Bill Cox, DDN VP of WW Channel Sales. "EPCC and its partner organisations have built a technologically advanced, state-of-the-art facility at the University of Edinburgh that opens a world of possibility to researchers across the UK. DDN is very pleased to join with OCF in assisting on this important project."
 
The storage environment built by OCF now uses:

• DDN Storage Fusion Architecture (SFA) 10K-X, an integrated storage appliance that maximises application performance while minimising TCO for big data, cloud, and content-intensive environments. It provides 7.8PB of useable storage capacity.
• IBM System Storage TS3500 Tape Library which is designed to provide an automated tape library for mainframe and open systems backup and archive in midrange to enterprise environments. It provides 19.5PB of capacity.
• 
  IBM GPFS software to enable storage capacity expansion to handle the growth of big data and digital information; improved efficiency through enterprise wide, interdepartmental file sharing; commercial-grade reliability to eliminate production outages and eases information life cycle management with policy-driven automation; cost-effective DR and BC; Active File Management to enable asynchronous access and control of local and remote files.