For information on all members of the Collection, search by Category, Company Name, Nominating Company, Application, Country or Keywords according to your area of interest.

Scalability/Technical Computing
Pittsburgh Supercomputing Center
Pittsburgh, PA
United States

Year: 2003
Status: Finalist
Category: Science
Nominating Company: Hewlett-Packard Company

Massive computing power accelerates the search for inexpensive polymers that would function the same way as simple but difficult-to-manufacture peptides that are known to be powerful anti-bacterials and that can reduce the incidence of in hospital infection.
A Unique Tool for Science

The Pittsburgh Supercomputing Center has developed and implemented
a unique and powerful computing system to advance engineering and
scientific research.

This system, called LeMieux (French for "The Best"), was developed by
PSC through a National Science Foundation grant to design and operate
a terascale computing system.

Terascale refers to computational power beyond a "teraflop" -- a trillion
calculations per second. While several terascale systems have been
developed for classified research at national laboratories, the PSC
system has been, from the date of its installation in October 2001, the
most powerful serving as an open resource for scientists attacking a wide
range of problems.

It is an innovative system, built from high-end microprocessors, which
leverages the reduced-cost benefit of commodity technology to produce
the highest performance for the available funding. It is the most powerful
system in the United States committed solely to public research.

The distinctiveness of LeMieux is how it differs from what are sometimes
called "purpose-built" systems, those designed from the ground up to be
supercomputers, and systems that, on the other hand, are relatively
simple, glued-together forms of clusters.

Distinguished from both these, LeMieux is a carefully-engineered
combination of hardware systems and system software designed for a
large market that includes high-end workstations and servers but that
nevertheless provides most of the cost-benefits of commodity technology
whose development is largely underwritten by sales into other markets.
Scientists who use it get the benefit of very powerful nodes, not built
specifically for scientific computing, but which are the best computational
nodes available. To take advantage of these nodes, the Pittsburgh
Supercomputing Center integrated them using the best available
interconnection hardware and with overall systems software designed for
the largest systems -- and which have a substantial amount of
engineering and development work invested in them.

This system represents an optimum blending of these various aspects
for the purpose of carrying out computational science. One might consider
more powerful, custom-built processor options, but they carry a huge cost.
Few components of this system are unique; however, as a whole it brings
together full software support, extremely powerful processors in extremely
strong nodes with a very low latency and high-bandwidth interconnect. At
an overall system level, the software binds all this together so that users
can treat it as a single entity.

Beyond this, LeMieux is distinguished from most other systems by its
scale -- 3,000 high-performance Alpha EV68 processors. By itself, this
scale confers unique capabilities on the system both in total processing
power (6 teraflops) and aggregate available memory (3 terabytes). It
enables most computations to be done much faster than other systems,
and this transforms the research paradigm in many important fields.

Development of this system drew on a history of collaboration between
PSC and Hewlett-Packard (then Compaq Computer Corporation), and
represents an extension of PSC's history of success at installing untried,
new systems -- resolving the myriad of unanticipated hardware and
software glitches that come up -- and rapidly making them available to the
scientific community as productive research tools.

Development and implementation of LeMieux, including its software and
networking, drew on fundamental research in computer science,
reflecting a significant strength of the Pittsburgh Supercomputing Center:
Its tri-partite affiliation with Westinghouse and with Carnegie Mellon
University and the University of Pittsburgh and the pooled
computing-related expertise of faculty and staff at both universities.

Exploiting the Tool:
New Weapons for the Germ Wars

Inexpensive polymers can extend the range of nature’s germ-fighter
arsenal. With hospital-acquired infection as the fourth-leading cause of
death, applications such as antiseptic operating tables and surgical
gowns provide a compelling imperative for research.

* * * *
This story begins with frogs. A lily pad is a fine place to perch in the
moonlight and broadcast mating calls if you’re a frog, but the picturesque
lily pond of a Monet painting is a cesspool of microorganisms. If you or I
swam in similar water, we’d risk our lives.

How do frogs manage to live happily in filthy water? It’s a simple question
that no one asked until the 1980s when it intrigued a scientist named
Michael Zasloff. In 1986 he found the answer — the skin of a frog harbors
armies of protein-like germ fighters. Zasloff isolated a molecule he
named magainin — Hebrew for "shield." Structurally like a protein only
smaller — just a few amino acids on a peptide chain — magainin has the
instincts of a secret service agent. Squads of these agents patrol a frog’s
skin, where they attack and destroy the cells of bacteria that threaten

The discovery of magainin launched a worldwide wave of research that’s
still going. Scientists have learned that plants and animals, including
some that lack the immune system of mammals, harbor a diverse
collection of defensive peptides, and they’ve learned that each species
has its own unique peptide arsenal, targeted to the bacteria, viruses and
other pathogens of that species’ environment.

The prospects to create new germ fighters from the model of mother
nature’s peptides are potentially boundless. Topping the list is the need
for new antibiotics that can defeat the protean ability of bacteria to resist
conventional penicillin-like antibiotics. That promise is especially
tantalizing in that a range of studies show that bacteria have little or no
ability to resist antimicrobial peptides.

Still other possibilities involve using molecules modeled on magainin
and its cousins to create germ-resistant materials such as bandages that
kill bacteria or toilet seats that sanitize themselves. With recent studies
citing hospital-acquired infection as the fourth-leading cause of death in
the United States, such possibilities and others — antiseptic operating
tables, surgical gowns, pillows and sheets — provide a compelling
imperative for research.

But the practical obstacles are huge. "The big catch," says University of
Pennsylvania computational scientist Michael Klein, "is you need 20 or 30
steps of organic synthesis to make these molecules, and you end up with
such high cost that it’s equivalent to grinding up diamonds."

Using LeMieux to Play Copycat with Mother Nature

Initial efforts to create a laboratory version of nature’s molecular germ
fighters led to several laboratory-synthesized peptides, one of which came
from Klein’s University of Pennsylvania colleague William DeGrado. While
these synthetic peptides represent a step forward, they also come with
built-in obstacles to practical use. "These natural peptides as well as their
synthetic analogues are expensive to prepare and difficult to produce on a
large scale," says DeGrado, "which limits their potential use."

Klein collaborated with DeGrado, providing theory-based molecular
simulations to help guide his laboratory work. In 2001 these two
scientists joined forces again to explore in a different, potentially more
practical direction. "We posed a question," says Klein. "Can we mimic the
peptide with something that’s cheap to make?" Based on initial results,
the answer appears to be yes.

To conduct this research, they needed a very powerful computing
resource – The Best. Klein and his co-workers turned to LeMieux to test
the possibilities of creating a polymer — an organic molecule easier to
make than peptides — structurally similar to magainin and with similar
germ-fighting ability.

With a series of computations on LeMieux, Klein and his colleagues were
able to develop an accurate computational model to forecast how a
magainin-like polymer would behave in the cellular environment. With
design guided by these computations, DeGrado’s team synthesized a
relatively simple polymer. Lab tests show the polymer has antibacterial
action similar to magainin and other peptides. The good news of this
work — reported in the Proceedings of the National Academy of Sciences
(April 2002) — is that a feasible new pathway is now open to extend the
range of mother nature’s anti-microbial arsenal.

Two-Faced Molecules

What do nearly 500 different natural germ-killer peptides have in
common? Like Janus, the Roman god of gates and doors, they face in
and out at the same time. All these small anti-microbes can, when
circumstances dictate, assume a shape in which, in chemical
terminology, they’re amphiphilic — one side of the molecule avoids water
(hydrophobic) while the other side likes it (hydrophilic).

This two-faced structure — scientists believe — is an essential part of
how nature equips peptides to destroy bacterial cells. While the
hydrophilic face turns out to the watery environs of the cell exterior, the
hydrophobic face can attach to lipids, the oil-like molecules that form cell
membranes, to pierce the membrane and create holes that eventually kill
the cell.

To create a polymer with similar properties, DeGrado and his colleague
Greg Tew approached Klein and asked him to model a class of polymers
with a fairly simple structure, called arylamides. "We were taking a further
step away from the protein backbone as the structural model," says
DeGrado, "which should give us good stability at reduced expense. We
had a shape that we thought should be compatible with the biological
activity we want. But this molecule could adopt many shapes, and the
question was whether it’s really happy in this shape, whether it’s
energetically favorable."

To answer this question, Klein’s group set out to simulate the arylamide
polymers in solutions that represent the cellular membrane and its
surrounding water, to see if they maintain structure and behave similarly
to magainin. The modeling tool for this job is molecular dynamics, a
method that tracks the shape and movement of a molecule and its
interaction with surrounding atoms. Most often used to model proteins,
molecular dynamics relies on "force fields" to represent the forces acting
between the atoms of the molecules. Initial attempts to model the
arylamide polymer showed that standard force fields gave inaccurate

The problem, the researchers surmised, was the uniqueness of the
arylamide backbone structure, chosen so that it wouldn’t freely rotate,
keeping hydrophobic side chains on one side and hydrophilic on the
other. To resolve this difficulty, Klein’s team carried out a series of highly
demanding quantum computations, using an approach called density
functional theory, to systematically derive accurate readings of the
rotational resistance of the arylamide backbone.

The researchers confirmed the accuracy of their revised molecular
dynamics model by simulating an arylamide structure and comparing it to
the actual structure from experiment. They then ran molecular dynamics
with several different versions of the arylamide polymer in an oil-water
solution. These simulations show the polymer moving toward the
oil-water interface and lodging there, mimicking the behavior of the natural
anti-microbial peptides.

Based on these results, DeGrado’s team synthesized the polymer and
tested its antibacterial properties. From the success of this work, the
University of Pennsylvania filed for several patents and created a
company, PolyMedix, to exploit the possible useful applications.

"We’ve identified a class of compounds," says Klein, "that the drug
industry would refer to as a possible lead compound. Some of these
short polymers are effective, but it will require systematic studies to
develop this further." While it’s only a first step, it’s a big one,
demonstrating not only the possibilities of using polymers to mimic
nature’s peptide germ fighters, but also how computational simulations
and laboratory experiment can work together to custom design molecules
for particular purposes.

The Pittsburgh Supercomputing Center produced and is operating
LeMieux, a unique, innovative and powerful computing system. It was
developed and installed successfully, on schedule, to provide United
States researchers in science and engineering an invaluable resource for
computational science.

The research by Klein and colleagues represents an exceptional case
study demonstrating how this resource has been of benefit to the United
States research community and, more broadly, how it produces new
knowledge that in concrete ways directly affects society and will improve
life for many.

Prior to LeMieux becoming available (early 2002), there was no similarly
powerful system available to researchers in the United States outside of a
few installations at classified government laboratory facilities. LeMieux
thus filled a large gap in United States research capability -- highlighted in
a 1999 report to President Clinton (The President’s Information
Technology Advisory Committee report).

When installed, LeMieux’s 3,000 parallel processors, capable of six
teraflops peak performance (six trillion calculations a second), provided
more than five times more computing capability than the next most
powerful system available to researchers through the National Science
Foundation. It has facilitated, and will continue to facilitate, progress in
many areas of significant social impact, such as the structure and
dynamics of proteins useful in drug design, storm-scale weather
forecasting, earthquake modeling, and modeling of global climate

The benefit of this system reflects that science is currently undergoing a
revolution as it experiences the benefit of information technology.
Computational science provides a third way of doing science,
complementary to theory and experiment, by using powerful computing
systems to create models and simulations that enact the predictions of
theory in a quantifiable form that can be directly tested against

As the strength of computational platforms increases, not only in raw
computational power, but also in communications, in memory, the
amount of data a system can hold and manipulate comfortably, new fields
for important research open up that were not feasible fields for work

The work of Klein and his colleagues to perform computational
simulations that predict the structure of magainin-like polymers (See
Introductory Overview) represents an important example of such work. The
potential benefits and useful applications of these polymers are manifold.
(See Success section.)
LeMieux, simply stated, is Information Technology. It leverages commodity
components and combines them at an unprecedented scale using
unique approaches, to produce a unique computational resource.

The particular scientific research cited could not have been carried out
without this information technology. Availability of LeMieux, the Pittsburgh
Supercomputing Center terascale computing system, was essential to
the work.

To determine whether it would be possible to create a polymer with
anti-microbial properties similar to natural defensive peptides, Klein and
colleagues at University of Pennsylvania needed to simulate a class of
polymers using computational techniques. Starting with a molecular
structure of an arylamide polymer, they created a computational model.

Their modeling tool, based in techniques of computational science, was
molecular dynamics, a method that tracks the shape and movement of a
molecule and its interaction with surrounding atoms. Molecular dynamics
relies on "force fields" to represent the forces acting between the atoms of
the molecules. Initial attempts to model the arylamide polymer showed
that standard force fields were unsuited for these molecules, giving
inaccurate results.

It took extensive quantum computations to resolve this problem, using an
ab initio (from first principles) approach, based in quantum theory, called
density functional theory. In total, Klein’s team required about 60,000
hours of computing time on LeMieux’s processors, to derive the
necessary force fields. This is the equivalent to over four years of
continuous computing on a modern PC.

With this information, it became possible to carry out molecular dynamics
simulations that show the arylamide polymer acting in a way that predicts
its similarity to natural defense peptides. Subsequent laboratory work
confirmed the computational predictions.

This research demonstrates convincingly how computational simulations
can guide and complement laboratory experiment to achieve significant
scientific results.

The terascale computing system, LeMieux, is an innovative and original
computing architecture. There is no other system like it in the world.

The node components were selected to optimize the trade-offs between
cost and the many aspects of performance including processing speed,
memory size, memory bandwith (transfer time) and memory latency
(access time). The equipment linking the node components was not
designed to serve a system of this size thus a unique connection scheme
was required. For the large number of components, simple statistics
predicts an unacceptably high failure rate for the entire system. A unique
approach of integrated "hot spares" combined with "on-the-fly" spare
implementation was developed to provide high availability of the total

The original insight of the University of Pennsylvania research team lies in
posing the question whether it might be possible to duplicate the
properties of very expensive, difficult-to-synthesize peptide molecules with
polymers that are much less expensive to produce. The insight to ask this
question, along with the computational resources and laboratory skill to
follow it up, has created the very promising possibility of making
inexpensive polymers with antibacterial properties.

LeMieux is fully operational. Installation was achieved on schedule in
October 2001, despite many obstacles in shipping as a result of
unforeseen events in September. This was achieved through a strong
commitment from HP engineers and staff working side-by-side with
Pittsburgh Supercomputing Center staff.

The following are several representative statements from researchers
who have used LeMieux to advance their work:

"This machine has been absolutely fantastic, enabling us to do
calculations that were previously impossible."
-- John Joannopoulos, Massachusetts Institute of Technology

"Teraflop supercomputing is essential and critical to the viability of this
-- Omar Ghattas, Carnegie Mellon University, speaking about modeling
blood flow at the microstructural level.

"With this machine . . . we have quite an achievement in technology
development for our science."
-- Klaus Schulten, University of Illinois Urbana-Champaign, speaking of
LeMieux and molecular biology.

"We’ve made exceptional progress that wouldn’t have been possible
without this system."
-- Steve Gottlieb, Indiana University, speaking of lattice computations in
quantum chromodynamics.

The University of Pennsylvania formed a company, PolyMedix, to attract
investment to exploit the success of proving the feasibility anti-microbial
polymers. As stated in the PolyMedix prospectus: These polymers will
dramatically improve the practice of medicine, quality of life, and treat
life-threatening bacterial infections, all of which represent enormous
commercial opportunities with multi-billion dollar markets.

Bacterial infections and contamination are one of the most significant
medical problems in the world today. Strains of bacteria resistant to all
known antibiotics are emerging, raising the specter of large-scale
uncontrollable infections. Hospital-acquired infections afflict over 2.4
million and kill 100,000 people annually, and are now the fourth leading
cause of death in the United States. Total healthcare costs of these
infections are over $45 billion.

There are currently no safe and effective bacteriocidal agents available for
materials applications. Innovation in the development of new antibiotic
agents has stalled, and companies are scrambling to fill the void. The
world therapeutic antibiotic market alone is currently over $25 billion, and
growing. There is a vital need both for new classes of antibiotic drugs,
and for materials that would inhibit bacterial growth and reduce infections
in countless settings.

Potential biomedical devices and products include contact lenses, IV
tubes, medical gloves, syringes, bandages, catheters. Additional potential
applications include: Paints and coatings. Countertops. Food preparation
surfaces. This list is endless. Potential consumer products include
sponges, eating utensils, toilet seats, sheets and pillowcases.
The key challenge in the development of LeMieux was that of scale. The
unprecedented number of components required innovative solutions to
extend hardware and software capabilities far beyond their design limits.
Typically, connected systems comprise 32 to 64 nodes. The interconnect
hardware was designed for a maximum of 128 nodes. LeMieux was
designed to use, continuously and simultaneously 750 nodes. Simply
maintaining an accurate inventory of more than 15,000 individually
identified components as they were delivered presented a challenge. To
maximize performance, the length of interconnect wiring must be
minimized requiring complex calculations to determine the optimum
locations of equipment on the computer room floor. The resulting
compact layout of the 200 cabinets presented unique problems in the
design of the air cooling system. The most-basic compute node requires
seven cables for connections to power and communications. Anyone who
has a stereo system can imagine the task of routing and managing more
than 5,250 wires.

Among the difficulties in the initial design of LeMieux was the process of
analyzing the available and future processors at the time when it was
necessary to prepare a proposal and produce the best possible,
cost-effective plan to serve the research community. The Pittsburgh
Supercomputing Center conducted an extensive dialogue with the
research community and with the scientific computing industry.

Researchers ranked single-processor speed as the most important
single criterion, and in the overwhelming majority of benchmarks, the
Alpha technology outperformed other systems. HP/Compaq also clearly
exhibited their interest to work with Pittsburgh Supercomputing Center
and the research, computer-science community to add value to what they
already had, to make this a system that meets the needs of the
high-performance community.

Other vendors with quality technology seemed not to evince any
pronounced interest in this project beyond providing technology. But no
vendor had commodity technology that had already been "productized"
and tested at the scale needed for this project. HP/Compaq clearly stated:
"We want to work on this." They shared a recognition that there was
pent-up demand for this system -- the nation's researchers need it, and
we need to get it out on schedule.

There have been many technical difficulties along the way, overcome by
skilled, dedicated staff at the Pittsburgh Supercomputing Center working
in collaboration with HP.

The University of Pennsylvania research team overcame a serious
obstacle in their modeling work. Initial attempts at molecular dynamics
simulations of the arylamide polymer showed that the "force field" used in
the model wasn’t accurate for the polymer’s structure.

The problem had to do with the uniqueness of the arylamide backbone
structure, chosen so that it wouldn’t freely rotate, keeping hydrophobic
side chains on one side and hydrophilic on the other. To solve this
problem, Klein’s team carried out a series of demanding quantum
computations to systematically derive accurate readings of the rotational
resistance of the arylamide backbone. With about 60,000 hours of
computing time using 128 LeMieux processors, they derived the force
fields they needed.

The researchers confirmed the accuracy of their revised molecular
dynamics model by simulating an arylamide structure and comparing it to
the actual structure from experiment. They then ran molecular dynamics
with several different versions of the arylamide polymer in an oil-water
solution. These simulations show the polymer moving toward the
oil-water interface and lodging there, replicating the behavior of the natural
anti-microbial peptides.

Taking their cue from these results, the laboratory team synthesized the
polymer and tested its antibacterial properties. Based on the success of
this work, the University of Pennsylvania filed for several patents and
created a company, PolyMedix, to exploit the possibilities for useful