New Supercomputer at University of Kentucky Puts UK in Top 10 HPC

A new high-performance supercomputer that will help power the University of Kentucky‘s research mission has already put the school in the top 10 public universities for computing power. The supercomputer, named the William N. Lipscomb, Jr. High-Performance Supercomputing Cluster (Lipscomb Cluster) after UK alumnus and Nobel Laureate William N. Lipscomb, Jr., is three times more powerful than UK’s old machine.

"The University of Kentucky likes being ranked in the top ten," said UK President Lee T. Todd, Jr. "And this new piece of equipment will do that for UK in the world of supercomputing. More exciting than that, however, is what this cluster will mean to our researchers. This supercomputer will allow our world-class researchers to discover new solutions to the complex problems facing the Commonwealth, the nation and the world."

The Lipscomb Cluster is based on the new Dell PowerEdge C6100 High Performance Computing (HPC) server chassis. The PowerEdge C6100 was chosen for its high density, shared infrastructure and efficient power and cooling features. The installation of the cluster began in July. After weeks of work by Dell, the UK HPC team, and the UK Center for Computational Sciences, the new supercomputer cluster went live August 25. The team completed the transition from the old supercomputer to the new cluster with minimum research interruption and in a much shorter amount of time than the last supercomputer transition in 2007.

The Lipscomb Cluster is important for UK research projects, including the design of new materials for magnetic applications, understanding biological chemical processes, investigations of the origins of the universe and subatomic particles, development of new types of batteries, improvements in automobile manufacture, and design of improved pharmaceuticals.

"A first-class research university must have first-class research facilities," said John Connolly, director of UK’s Center for Computational Sciences. "Computation is becoming a more and more important component of modern research. The new UK supercomputer will provide high-speed access for our faculty and students so they can be competitive with anyone in the country, and will help UK achieve its goal of Top 20 status."

Matthew Beck, an assistant professor and researcher in the College of Engineering, said the computer will allow researchers to run calculations of increasingly realistic and complex systems.

"Metal-oxide nanoparticles, one of the things we’re using the Lipscomb Cluster to study now, are a great example," said Beck. "They’re exciting because they have shown both the potential to protect against oxidative stress-related diseases like Alzheimer’s and as high-performance catalysts for the production of molecular hydrogen and oxygen for fuel cells. But nobody really understands how these nanoparticles work. Ultimately, unless we can calculate properties of whole nanoparticles, we don’t really know if what we’re calculating is what really happens. The Lipscomb Cluster allows us to do exactly that. Suddenly the line between traditional experiment – which is about the real things that you can hold in your hand, and computation – which is supposed to be just theory, goes away."

During benchmark testing, the Lipscomb Cluster was measured at speeds of over 40 teraflops. A teraflop is a trillion calculations, known as floating point operations, per second. This would rank UK 162 on the current list of the top 500 supercomputers in the world, and eighth among higher education sites not funded by government grants. (Though the rankings have not been updated since UK’s new high-performance computer went live, its speed would currently place it at these levels among other supercomputers.) The previous supercomputer measured speeds under 13 teraflops, making the new machine more than three times faster.

"We’re thrilled the University of Kentucky chose Dell’s HPC solutions to power their research capacity expansion goals," said John Mullen, Dell vice president and general manager for higher education. "The University’s timeline for implementation with minimal disruption to researchers is among the most aggressive and successful deployments we have witnessed in the higher education HPC community. This is a testament to UK’s deep HPC experience and the collaborative relationship UK fosters with its vendors."

The Lipscomb Cluster offers 376 Basic Nodes, each powered by two 2.66 GHz Intel Xeon X5650 (Westmere) CPUs (4,512 processor cores total) and 36 GB of RAM (more than 13 TB RAM total). For comparison, a high-end desktop computer, like the Dell Studio XPS 8100, uses a processor similar to the two in each of UK’s compute nodes, but the cluster has 752 of these connected together by a high speed fabric. The cluster also has eight Hi-Mem Nodes, each powered by four 2.66 GHz Intel Xeon X7560 CPUs (256 processor cores total) and 512 GB of RAM (4 TB total). According to Beck, this equates to about 2,800 times the computing power of the average personal laptop, such as the Intel i3-350M laptop processor available in laptops such as the Dell Studio 15 and 17.

The compute nodes are networked with a Mellanox quad data rate Infiniband switch. The high-performance Panasas cluster file system contains about 200TB of usable disk storage (for home and scratch directories). The cluster includes a dedicated backup node running IBM’s Tivoli Storage Manager (TSM) and Hierarchical Storage Manager (HSM) with a 10 gigabit link to the rest of UK’s central backup system. This will cut cluster backup times by an order of magnitude and drastically improve access to UK’s near-line tape storage system, which is heavily used by UK’s supercomputing researchers. The Lipscomb Cluster provides approximately a three-fold increase in processing power and a five-fold increase in data storage, while reducing overall power consumption.