ACES wild

NSF-sponsored program brings unprecedented capabilities to scientists studying natural world

A picture's worth a thousand numbers. For researchers who model complex natural processes such as weather patterns or groundwater movement, translating data into effective displays is often a challenge. Here, Vanda Grubisic shows an animated three-dimensional image depicting her work modeling airflow over the European Alps. (Photo by John A. Benson, courtesy of Silicon Graphics, Inc.)

DRI atmospheric scientist Dr. Vanda Grubisic has been doing a little... remodeling.

Her budget? A $3.4 million grant from the National Science Foundation's Experimental Program to Stimulate Competitive Research, or EPSCoR.

Her objective? To create the Advanced Computing in Environmental Sciences, or ACES, Program—a collaborative network of high-end computer resources, support services and communication technology to enhance environmental research at DRI, the University of Nevada, Reno and the University of Nevada, Las Vegas.

The end result? A new level of excellence for computational environmental science research—including 3-D data visualization—and education in the state of Nevada.

"Environmental science disciplines are very data-intensive, and we saw this as an opportunity to advance our capabilities greatly and become more competitive in computer modeling, data visualization and advanced computing techniques," says Grubisic, who serves as the statewide coordinator for ACES.

Examples of such data-intensive research abound at UNR, UNLV and DRI and include Grubisic's own work modeling airflow over mountainous terrain; studies of groundwater flow; climate data analysis and modeling; and applications of geographic information systems and remote sensing.

ACES was launched two years ago and in that time, Grubisic has been busy bringing together science and information technology experts from around the state, acquiring key pieces of new equipment and hiring some help in the form of UNIX System Administrator Mark Ballew and Scientific Applications Programmer Shulan Liu.

A key accomplishment has been the creation of the core of the Nevada Environmental Computing Grid, a statewide distributed computational infrastructure—accessible from DRI campuses in Reno and Las Vegas, and from UNR and UNLV—that will give researchers some seriously pumped-up computing power.

Move over, HAL. This shared-memory high-performance supercomputer is the core of ACES computing power. Thanks to the SGI Altix 3700, Nevada scientists are able to perform large computing tasks in record time. (Photo by John A. Benson, courtesy of Silicon Graphics Inc.)

"The ACES computing grid will allow them to spread their computations over multiple machines and run large jobs—much larger than they could run on any single machine—on the grid by tapping into unused computational resources shared on the grid," Grubisic explains.

At the heart of ACES grid computing power is the SGI Altix 3700 from Silicon Graphics. Housed at DRI in Reno and on-line since October 2003, the shared-memory high-performance supercomputer boasts 40 Intel Itanium2 CPUs, 80 Gigabytes of RAM, 3 Terabytes of disk space and the Linux kernel.

In short, it's big, it's fast and it's an excellent platform for running scientific applications. "Currently," Grubisic says, "there are about 40 users of this machine and that's growing all the time."

One of those users is DRI's Dr. Darko Koracin, whose high-resolution weather forecasting model, known as MM5, is supported by the machine. "We have been using this computer system to model the diurnal evolution of the cloudy marine atmospheric boundary layer on five nested model domains with the highest horizontal grid resolution of 1 kilometer," Koracin explains, "which was not practical and in some cases even impossible to perform on the previous generation of our computers. As an illustration, just the innermost domain of the MM5 setup has approximately five million grid points on which to compute all prediction variables—winds, temperature, humidity and turbulence kinetic energy—every three seconds."

DRI's Dr. Alison Murray, whose work focuses on microorganisms found in extreme environments, is a believer, too. "The ACES program has introduced my research group to parallel computing and provided us with the ability to process in hours bioinformatics data that previously would have taken weeks."

Another key feature of ACES is a collaboration grid—a system of sophisticated multimedia conferencing technology—built around Access Grid^TM technology developed by the National Center for Supercomputing Applications and maintained by the Futures Laboratory at Argonne National Laboratory.

"Think of it," Grubisic says, "as video conferencing on steroids. It definitely offers something beyond video conferencing and allows Nevada scientists to hold large-scale distributed meetings and collaborative work sessions as well as share seminars, lectures and tutorials involving Access Grid Nodes in and outside the state."

With the ability to share data in real time and display video images of many different locations at once, it basically makes users feel as if they're in the same room as their colleagues, even though they are thousands of miles apart.

Dominated by an immense 17-foot by 4.5-foot acrylic screen, and sporting hidden data pop-ups, wireless Internet, strategically placed cameras and seating with a commanding view, DRI's Access Grid Node in Reno bears more than a passing resemblance to a familiar Star Trek set. It's a feeling enhanced when small windows materialize on the wall-to-wall screen, beaming you to some of the more than 400 locations accessible from this one room.

"In effect," Grubisic says, "you can connect with as many points as you want, limited only by your bandwidth."

The ACES grant funded three conference-room sized Access Grid Nodess: the facility at DRI in Reno, one in UNR's computer science department and one in the computing center at UNLV. Their value, Grubisic says, was quickly realized.

"Since then, DRI has added a node to its campus in Las Vegas, UNR has added one at its medical school and UNLV has added one in its main library. That's zero to six sites in just two years."

Nevada, she says, is just the kind of place to benefit from this technology. "This is quite an advantage in a geographically spread-out state, allowing researchers to collaborate without leaving their home institutions. In addition, all the federal funding agencies have nodes, making it a nice way to connect with program managers and save on trips to Washington."

And, the Access Grid Node at DRI in Reno houses a little something extra. The Scientific Visualization Laboratory, or VisLab, is a facility designed to bring the inherent complexities of environmental studies to life with remarkable clarity—that is to say big, animated and even three-dimensional.

Three Digital Light Processing^TM projectors, driven by a high-performance visualization computer cluster, can cast individual images onto the large screen or fill the entire 17 feet with a single image.

"For researchers who use satellite imagery or Geographic Information Systems," Grubisic explains, "this gives them an unprecedented capability to examine large areas at a time with incredible resolution."

A front-projection system displays three-dimensional images on a smaller drop-down screen, a feature that gives computer modelers a host of new options for interpreting their data.

"A real challenge for environmental scientists whose work is based on tremendous amounts of data is sifting through all that and finding a way to display it effectively," Grubisic says. "For most visualization purposes we have had to use two dimensions—paper, or a computer screen—yet we are modeling three-dimensional processes."

In the VisLab, the only such facility in Nevada, computer simulations can not only be displayed in all three-dimensions, they can be animated, manipulated and viewed from practically infinite perspectives.

A demonstration of these capabilities, complete with stylish 3-D glasses, shows you how clouds and air currents flow over the Alps, takes you beneath the Earth's tectonic plates to pinpoint quake epicenters and lets you ride in the eye of a hurricane.

Sure, it's entertaining, but beyond that Grubisic says it gives Nevada researchers a whole new way of looking at their work. "With these visualization capabilities I am able to gain new physical insights into the processes I am studying and to discern much finer detail."

Of course, technology is nothing without people to use it, and ACES is dedicated to helping researchers take full advantage of the new capabilities at their disposal.

"One of our objectives is to help them in their choice of computing platform and to find the best ways to use ACES to further their research," she says. The program has also awarded seven graduate fellowships, four postdoctoral fellowships and four seed grants in the past year, with seven additional graduate fellowships awarded this year.

The ACES computing grid and collaboration grid put Nevada on the leading edge but it won't be long, Grubisic says, before other states join in.

"Collaborative research environments... focused around regional and/or discipline-oriented centers, such as our Nevada Environmental Computing Grid, are relatively rare now, but are the likely direction that the NSF-supported IT infrastructure development will take in the future."

With the computing grid up and running, and the Access Grid Nodes and VisLab completed, Grubisic is ready to unveil her work to the world with a fall grand opening. Researchers, on the other hand, don't stand on ceremony and are already putting their new capabilities to good use.

"Just since January 2004," Grubisic says, "there has been more than $3 million in competitive proposals produced that plan on using ACES facilities in their research. That's something of which we are very proud."

-Jackie Allen