NCAR Launches 5.34 Petaflop Cheyenne System Built by SGI/HPE and DDN

The National Center for Atmospheric Research (NCAR) is launching operations this month of one of the world’s most powerful and energy-efficient supercomputers, providing the nation with a new tool to advance understanding of the atmospheric and related Earth system sciences.

Scientists used the Yellowstone supercomputer to develop this 3-D rendering of a major thunderstorm in July 2011 that caused flooding in Fourmile Canyon west of Boulder. The colors show conditions in the clouds, including ice particles (light blue), graupel (orange), snow (pink), rain (blue), and water (grey). (Image by David Gochis, NCAR)

Named Cheyenne, the 5.34-petaflop system is capable of more than triple the amount of scientific computing performed by the previous NCAR supercomputer, Yellowstone. It also is three times more energy efficient.

Scientists across the country will use Cheyenne to study phenomena ranging from wildfires and seismic activity to gusts that generate power at wind farms. Their findings will lay the groundwork for better protecting society from natural disasters, lead to more detailed projections of seasonal and longer-term weather and climate variability and change, and improve weather and water forecasts that are needed by economic sectors from agriculture and energy to transportation and tourism.

“Cheyenne will help us advance the knowledge needed for saving lives, protecting property, and enabling U.S. businesses to better compete in the global marketplace,” said Antonio J. B,lacchi, president, the University Corporation for Atmospheric Research. “This system is turbocharging our science.“

UCAR manages NCAR on behalf of the National Science Foundation (NSF). Cheyenne ranks as the 20th fastest supercomputer in the world, and the fastest in the Mountain West. It is funded by NSF as well as by the state of Wyoming through an appropriation to the University of Wyoming.

Cheyenne is housed in the NCAR-Wyoming Super-computing Center (NWSC), one of the nation’s premier super-computing facilities for research. Since the NWSC opened in 2012, more than 2,200 scientists from more than 300 universities and federal labs have used its resources.

“Through our work at the NWSC, we have a better understanding of such important processes as surface and subsurface hydrology, physics of flow in reservoir rock, and weather modification and precipitation stimulation,” said William Gern, VP of research and economic development, University of Wyoming. “Importantly, we are also introducing Wyoming’s school-age students to the significance and power of computing.“

The NWSC is located in Cheyenne, WY, and the name of the new system was chosen to honor the support the center has received from the people of that city. The name a, commemorates the upcoming 150th anniversary of the city, which was founded in 1867 and named for the American Indian Cheyenne Nation.

Increased power, greater efficiency
Cheyenne was built by Silicon Graphics International, Corp. (SGI), now part of Hewlett Packard Enterprise, with DataDirect Networks, Inc. (DDN) providing centralized file system and storage components.

Cheyenne is capable of 5.34 quadrillion calculations per second (5.34 petaflops, or floating point operations per second).

The new system has a peak computation rate of more than 3 billion calculations per second for every watt of energy consumed. That is three times more energy efficient than the Yellowstone supercomputer, which is also highly efficient.

The storage system for Cheyenne provides an initial capacity of 20PB, expandable to 40PB with the addition of extra drives. The DDN system also transfers data at the rate of 220GB/s. which is more than twice as fast as the previous file system’s rate of 90GB/s.

Cheyenne is the latest in a long and history of supercomputers supported by the NSF and NCAR to advance the atmospheric and related sciences.

“We’re excited to prov, the research community with more super-computing power,” said Anke Kamrath, interim director,NCAR’s Computational and Information Systems Laboratory, which oversees operations at the NWSC. “Scientists have access to increasingly large amounts of data about our planet. The enhanced capabilities of the NWSC will enable them to tackle problems that used to be out of reach and obtain results at far greater speeds than ever.“

More detailed predictions
HPC such as Cheyenne allow researchers to run increasingly detailed models that simulate complex events and predict how they might unfold in the future. With more super-computing power, scientists can capture additional processes, run their models at a higher resolution, and conduct an ensemble of modeling runs that provide a fuller picture of the same time period.

“Providing next-generation super-computing is vital to better understanding the Earth system that affects us all,” said NCAR director James W. Hurrell. “We’re delighted that this powerful resource is now available to the nation’s scientists, and we’re looking forward to new discoveries in climate, weather, space weather, renewable energy, and other critical areas of research.“

Some of the initial projects on Cheyenne include:

• Long-range, seasonal to decadal forecasting: Several studies led by George Mason University, the University of Miami, and NCAR aim to improve prediction of weather patterns months to years in advance. Researchers will use Cheyenne’s capabilities to generate more simulations of finer-scale processes in the ocean, atmosphere, and sea ice. This research will help scientists refine computer models for improved long-term predictions, including how year-to-year changes in Arctic sea ice extent may affect the likelihood of extreme weather events thousands of miles away.
• Wind energy: Projecting electricity output at a wind farm is extraordinarily challenging as it involves predicting variable gusts and complex wind eddies at the height of turbines, which are hundreds of feet above the sensors used for weather forecasting. University of Wyoming researchers will use Cheyenne to simulate wind conditions on different scales, from across the continent down to the tiny space near a wind turbine blade, as well as the vibrations within an individual turbine itself. In addition, an NCAR-led project will create high-resolution, 3-D simulations of vertical and horizontal drafts to provide more information about winds over complex terrain. This type of research is critical as utilities seek to make wind farms as efficient as possible.
• Space weather: Scientists are working to better understand solar disturbances that buffet Earth’s atmosphere and threaten the operation of satellites, communications, and power grids. New projects led by the University of Delaware and NCAR are using Cheyenne to gain more insight into how solar activity leads to damaging geomagnetic storms. The scientists plan to develop detailed simulations of the emergence of the magnetic field from the subsurface of the Sun into its atmosphere, as well as gain a three-dimensional view of plasma turbulence and magnetic reconnection beneath the Sun’s surface that lead to plasma heating.
• Extreme weather: One of the leading questions about climate change is how it could affect the frequency and severity of major storms and other types of severe weather. An NCAR-led project will explore how climate interacts with the land surface and hydrology over the US, and how extreme weather events can be expected to change in the future. It will use advanced modeling approaches at high resolution (down to just a few miles) in ways that can help scientists configure future climate models to better simulate extreme events.
• Climate engineering: To counter the effects of heat-trapping greenhouse gases, some experts have proposed artificially cooling the planet by injecting sulfates into the stratosphere, which would mimic the effects of a major volcanic eruption. But if society ever tried to engage in such climate engineering, or geoengineering, the results could alter the world’s climate in unintended ways. An NCAR-led project is using Cheyenne’s computing power to run an ensemble of climate engineering simulations to show how hypothetical sulfate injections could affect regional temperatures and precipitation.
• Smoke and global climate: A study led by the University of Wyoming will look into emissions from wildfires and how they affect stratocumulus clouds over the southeastern Atlantic Ocean. This research is needed for a better understanding of the global climate system, as stratocumulus clouds, which cover 23% of Earth’s surface, play a key role in reflecting sunlight back into space. The work will help reveal the extent to which particles emitted during biomass burning influence cloud processes in ways that affect global temperatures.