Purdue University With Anvil HPC

Purdue University’s HPC power has leveled up in strength and speed with its new Anvil supercomputer.

Pictured with the Anvil HPC are from left to right: Rajesh Kalyanam, research scientist; Carol Song, senior research scientist; Xiao Zhu, senior research scientist; and Preston Smith, executive director, Purdue Research Computing

The new capacity will allow Purdue to contribute to the nation’s 21st- century research agenda by simultaneously powering scientific computational and data-driven tools at multiple universities.

Having begun operations in November, Anvil is at full capacity, funded by a $22 million grant from the National Science Foundation. In its mission to serve as a resource for Purdue and outside institutions throughout the US, it already has facilitated work from over 20 institutions in its first four months of operation.

“Anvil is not only the largest capacity system Purdue has ever built, it’s the most diverse,” said Carol Song, senior research scientist for ITaP research computing and principal investigator and project director, Anvil. “Its various components are all integrated in one place with GPUs and large memory nodes complementing a cluster of 1,000 compute nodes. As Anvil grows, it is also able to take on much more heterogeneous workflows that are more common in the research happening today at Purdue.“

As evidenced by its use thus far, Anvil’s next-level strength and versatility promises to be a boon for Purdue researchers who have identified myriad potential uses for its computing power.

For example, Jonathan Poggie, professor of aeronautics and astronautics, is looking to use Anvil to speed up his study on the prediction and eventual control of the aerodynamic heating produced in flight at high Mach speeds. In addition, Daniel Aliaga, associate professor of computer science, and his research group are exploring the use of Anvil’s GPU capability to help facilitate the creation of what-if design tools in digital city planning. These futuristic tools will enable urban planners worldwide to automatically integrate, process, analyze and visualize complex interdependencies among urban form, function and the natural environment.

Additionally, new treatments for heart disease are the promise of a project that came to Anvil from out of state. Yinglong Miao, an assistant professor in the department of molecular biosciences and center for computational biology at the University of Kansas, works in advanced biomolecular modeling and drug discovery. He is using Anvil’s power to accelerate his biomolecular simulations to unlock remedies for heart disease.

“Using CPUs this work could take months, if not years,” he said. “With the GPUs on Anvil, we can run these simulations much faster. Instead of needing months, we just need a couple of weeks.”

Anvil’s current strength is in a category that challenges the imagination. At peak performance, it can operate at a 5.1 petaFLOPS (floating-point operations per second). A computer system with one petaFLOPS is capable of performing one quadrillion FLOPS. A one-petaFLOPS computer system can manage in one second what would take an individual a calculation every second for 31,688,765 years. Multiply those individual calculations by a factor of 5.1, and the product is Anvil’s single-second peak capacity.

Anvil’s new power will increase the computing capacity available to users of the NSF’s Extreme Science and Engineering Discovery Environment (XSEDE), in which Purdue has been a partner for the past 10 years. It will be able to deliver over 1 billion computing core hours to XSEDE each year. In addition, 10% of HPC power will be made available for high-impact initiatives including research partnerships with industry.

Researchers may request access to Anvil via the XSEDE allocation process. New allocations may be requested at any time by filling out a request form via the XSEDE user portal. Allocation on Anvil for class use and training may be requested via the education allocation process throughout the year. Large research allocations can be requested four times a year (see timetable) and are peer-reviewed quarterly.

Anvil’s details
Anvil consists of 1,000 nodes, which are individual computers that consist of one or more central processing units (CPUs) together with memory. Each node has 2×64-core, third-generation AMD EPYC processors and will deliver more than 1 billion CPU core hours to XSEDE every year. Anvil’s nodes are interconnected with 100GB/s Mellanox HDR InfiniBand. The supercomputer ecosystem also includes 32 large memory nodes, each with 1TB of RAM, and 16 nodes each with 4 NVIDIA A100 Tensor Core GPUs (graphics processing units) providing 1.5 petaFLOPS of single-precision performance to support ML and AI applications. Research on Anvil will be able to leverage a diverse set of storage technologies, anchored by a 10PB+ parallel filesystem, boosted with over 3PB of flash disk. Novel workflows will benefit from block and object storage systems also supported by Anvil.