The Computerworld Honors Program
Honoring those who use Information Technology to benefit society
LOCATION:
Carmel, IN, US

YEAR:
2007

STATUS:
Laureate

CATEGORY:
Environment, Energy and Agriculture

NOMINATING COMPANY:
Hitachi

ORGANIZATION:
Midwest Independent Transmission System Operator

PROJECT NAME:
Midwest Independent Transmission System Operator Maintains 100 percent Uptime Using a Joint Storage Solution from Hitachi Data Systems and Sun

Short Summary
The Midwest Independent Transmission System Operator (Midwest ISO) is the nation's first Regional Transmission Organization (RTO) approved by the Federal Energy Regulatory Commission (FERC). It is responsible for operating the transmission system that delivers electricity from generating plants to the load-serving entities that serve residential and commercial customers.

The Midwest ISO’s two main business objectives are maintaining and improving electric system reliability in the Midwest and managing a wholesale energy market. To meet increasingly stringent government regulations, Midwest ISO replaced its existing storage platform with a Hitachi Data Systems (HDS)/Sun TagmaStore solution.

Together, the HDS/Sun solution enabled Midwest ISO to implement a tiered storage solution and deliver a highly available uptime of mission-critical applications. As a result, the Midwest ISO is the first US-based Independent Transmission Systems Operator to generate dispatch signals for 1,500 generators every five minutes based on highly-sophisticated algorithms that determine the optimal least cost power generation based on system transmission and generation constraints.

The criticality of this storage system cannot be overemphasized. The system supports the applications that the Real-Time Operators use to manage both the transmission grid and 1,500 generating units to literally “keep the lights on” in 14 states across the Midwest including one province in Canada. Performance, availability and reliability are of paramount importance since the generation of power must match the load at all times to maintain the reliability of the grid. The application systems that are supported by the storage system are critical to Midwest ISO’s Real-time Operators. These individuals make minute-to-minute operations decision that have a direct impact on the reliability of the transmission network. The basis for their decisions is the data, analysis and simulations that are performed every few minutes by the real time application systems, without which they are blind to the conditions of the electrical network.

Introductory Overview
The Midwest ISO is based in Carmel, Indiana, and is responsible for monitoring and operating the electric transmission system that delivers power from generating plants to wholesale power transmitters (the entities that deliver power eventually through the transmission grid to distribution companies that, in turn, deliver power to residential and commercial customers). The Midwest ISO's primary roles are to ensure equal access to the transmission system and to maintain or improve electric system reliability in the Midwest and parts of Canada.

The Midwest ISO was founded on Feb. 12, 1996, and was specifically formed to comply with FERC's concept of an independent organization that will ensure the smooth regional flow of electricity in a competitive wholesale marketplace. The Midwest ISO's primary objective is to reliably "direct traffic" on the wholesale bulk electric power transmission lines. In this role, the Midwest ISO ensures that every electric industry participant has access to the lines and that no entity has the ability to deny access to a competitor. The Midwest ISO also manages the use of the lines to make sure that they don't become overloaded. The Midwest ISO manages more than 93,600 miles of transmission lines covering 920,000 square miles from Manitoba, Canada, to Kentucky.

Until the recent project overhaul, the Midwest ISO had been with another tier 1 storage competitor, which supported their production and back-up environments. However, the competitor’s hardware required planned outages for bin file changes and microcode updates. Additionally unplanned outages for additional new hardware were also necessary. As the core of the Midwest ISO’s business is about uptime and providing 24x7 service to power customers throughout the Midwest, it decided upon a combined solution from Sun and Hitachi Data Systems. This has allowed a partnership for the SAN and the NAS changes that Midwest ISO has migrated.

To begin, the Midwest ISO deployed a Hitachi Data Systems TagmaStore 9990 at both their Production and Backup facilities in doing so; the Midwest ISO leveraged Hitachi’s TrueCopy Heterogeneous Remote Replication Software, which provides real-time replication of data between the two facilities. The organization also implemented Hitachi’s ShadowImage Heterogeneous In-System Replication Software to create multiple, real-time production copies of data within the same subsystem, which are then used for testing and backup.

The Midwest ISO also utilized Hitachi’s HiCommand Suite of storage and performance management tools, including HiCommand Device Manager and HiCommand Tuning Manager; to manage both of its TagmaStore 9990s located in Indiana, as well as their facility in Minnesota where they have installed a Hitachi Data Systems/Sun TagmaStore 9985, from a single location.

Finally, the Midwest ISO attached its previously-purchased storage subsystem to the Hitachi Data Systems/Sun TagmaStore 9990. This virtualized the other storage subsystem and allowed the Midwest ISO to protect its previous storage investment by leveraging its capacity for test environments.

Since the Hitachi Data Systems implementation, the Midwest ISO has gained better through-put and latency capabilities to meet level requirements dictated by their stakeholders they are seeing higher performance, faster throughput, and reliability of their storage infrastructure, which further adds to the ability of keeping the Midwest electricity grid up and running.

In short, the Midwest ISO is experiencing better returns on their storage architecture. It has achieved several technological benchmarks (detailed in the technology section) which have not only allowed the operators to improve accuracy in their runs but have also indirectly provided enhanced service to millions of consumers across the Midwestern region.


Benefits
Has your project helped those it was designed to help?   Yes

What new advantage or opportunity does your project provide to people?
The Midwest ISO is an essential link in the safe, cost-effective delivery of electricity across much of North America. Maintaining reliability of the wholesale bulk electric system in the Midwest is the core responsibility of the Midwest ISO, and the decision of its IT department to change the company’s existing storage infrastructure will play a significant role in helping the Midwest ISO deliver on this promise.

Current requirements dictate 99.99 percent uptime of the Midwest ISO’s systems, an average of less than one hour or less of downtime per year. With the Midwest ISO’s previous storage-based system, meeting this goal was not possible. With the implementation of the new Hitachi-based system, however, the Midwest ISO has been able to achieve an expected highly available up-time, making their system one of the most reliable on the market today.

Additionally, through the benefits delivered by a tiered storage system, the Midwest ISO has been able to reduce the overall cost of managing a 200TB storage system that is expected to double in the next two years. Most importantly though, the Midwest ISO project delivers a more reliable electric grid for some 40 million inhabitants in 14 states and part of Canada.


Has your project fundamentally changed how tasks are performed?   Yes

How do you see your project's innovation benefiting other applications, organizations, or global communities?
The Midwest ISO storage overhaul project has been influential by providing similar organizations throughout the world a standard by which to measure and compare the reliability of their IT infrastructures. Several pieces of the Midwest ISO solution have been considered “World Class Best” Korea and China have come to the Midwest ISO to learn about managing the electric grid looking for solutions with hopes of mirroring performance and results. Singapore has also expressed an interest in learning more about the Hitachi Data Systems implementation and the benefits it offers.

The Importance of Technology
How did the technology you used contribute to this project and why was it important?
The Hitachi Data Systems technology used in the storage infrastructure overhaul was significant to the project in the differentiation it offered in terms of reliability, support and performance.

Because of the upgrade from an older technology of another vendor’s to a Hitachi Data Systems storage array, the Midwest ISO has reduced data transaction time for calculating generation unit dispatch instructions from 120 seconds to 90 seconds, an improvement of 25 percent. Before the change, transaction times of two minutes made it difficult to run multiple scenarios for control operators to have sufficient analysis time to select the optimum scenario.

Additionally, the advancement in storage array technology over the past decade has been instrumental. Ten years ago, it would not have been possible to do what the Midwest ISO is doing today. Now, every five minutes, the Midwest ISO is able to dispatch signals to 1,500 generators, telling them how much power they should be generating. Midwest ISO is the only ISO in the U.S. accomplishing this feat in five minutes, with all others taking 15 minutes or more to do the same thing—and the Midwest ISO credits their storage array as a critical key to achieving this objective.


Originality
What are the exceptional aspects of your project?
The Midwest ISO had to combine the disparate operating systems of Tru-64, Linux, Windows and Solaris into a working migration effort without experiencing any loss of service. The volume of servers equated to over 700 attached to the SAN and NAS spread out across 5 environments in three data centers.

The server and storage environment at Midwest ISO is complex and sophisticated. There are over 7 environments and multiple operating systems which rely on the storage infrastructure, including Tru-64, Linux, Windows and Solaris. Failure was not an option and the migration had to occur while maintaining our 99.9% availability levels. Over 700 servers 700 attach to the SAN and NAS environments and are located in three data centers.

The Midwest ISO was able to accomplish this migration into the new Hitachi array only by working in concert with a combined partnership of Sun and Hitachi and its exceptional internal IT workforce. Together, the group manages a project that at one time involved over 75 resources to make the migration from the old system to the new system possible. The Midwest ISO was able to migrate their SAN into the new storage platform in the hottest part of the summer while having no unplanned downtime because of this migration. All of this is attributed to a great partnership with our vendors and an incredible project effort of planning, skill and communication.


How is it original?
The Midwest ISO is the first of the ISOs and RTOs that manage large portions of the electrical grid to provide solutions for issuing generation dispatch signals every 5 minutes. The reliability of data to execute complex algorithms with large amounts of data is what makes companies like the Midwest ISO unique and different. Missing one set of 5 minute generation signals can potentially reap havoc on the electric grid (depending upon the other events occurring on the grid.) The Midwest ISO is the largest company in the United States involved in this kind of activity. Without a reliable SAN or NAS environment, the Midwest ISO can not manage the electric grid.

Accomplishing this migration effort without any loss of service to the grid is what set this effort apart from many other storage migration efforts.

Through this effort, the Midwest ISO is able to now take advantage of:
-- Seamless server virtualization
-- Simplified data tiering
-- Confident microcode upgrades


Is it the first, the only, the best or the most effective application of its kind?   First

Success
Has your project achieved or exceeded its goals?   Exceeded

Is it fully operational?   Yes

How many people benefit from it?   40 mil

If possible, include an example of how the project has benefited a specific individual, enterprise or organization. Please include personal quotes from individuals who have directly benefited from your work.
920,000 square miles of territory served with an approximate population of 40 million.

The migration to the new storage platform has made a significant impact on Midwest ISO’s Operations Center to quickly and smoothly manage the electrical transmission network in our footprint. Specifically the new storage platform improved performance by 20% for the network simulation and optimization calculations that we perform every 5 minutes that are fundamental to our operations. In addition, the reliability of the new storage platform has been extremely reliable with no downtime. Reliability is our business since we are a real-time 24X7X365 operation that cannot tolerate any system slowdown or failure. The transition to the new storage platform was also impressive with no impact on our operations during this critical period.” Allen Phelps, Director of Real-Time Operations.

How quickly has your targeted audience of users embraced your innovation? Or, how rapidly do you predict they will?
Due to the smooth implementation and seamless transition to the HDS equipment, the user community’s confidence in IT has significantly increased. They became even more comfortable with IT’s ability to make changes of this magnitude on the fly.

Difficulty
What were the most important obstacles that had to be overcome in order for your work to be successful? Technical problems? Resources? Expertise? Organizational problems?
Due to the smooth implementation and seamless transition to the HDS equipment, the user community’s confidence in IT has significantly increased. They became even more comfortable with IT’s ability to make changes of this magnitude on the fly.

Often the most innovative projects encounter the greatest resistance when they are originally proposed. If you had to fight for approval or funding, please provide a summary of the objections you faced and how you overcame them.
The IT team at the Midwest ISO did face obstacles that had to be overcome before a successful overhaul of their storage infrastructure could be realized. As is the case with most non-profit organizations globally, the Midwest ISO budget is well managed and expenditures of this magnitude are well thought out and evaluated prior making this level of commitment in behalf of our membership. Another issue faced was the correction of “start-up” roots, meaning… MISO has developed very fast from 50 people in 2000 to over 650 in 2006, MISO effectively outgrew its prior storage solution, but the importance of what MISO does continued to grow exponentially. Outages were not acceptable from our stakeholders perspective. From a technology stand-point, the Midwest ISO also realized that the Hitachi Data Systems Network Attached Storage (NAS) blade would not meet all of the Midwest ISO’s requirements. They needed a quick replacement to avoid double payment for the new HDS equipment and monthly maintenance on their previous equipment. To remedy this issue, Hitachi partner Sun Microsystems committed to replacing the Hitachi Data Systems NAS Blades with their own NAS appliances at no charge to the Midwest ISO.
Digital/Visual Materials
The Program welcomes nominees to submit digital and visual images with their Case Study. We are currently only accepting .gif, .jpg and .xls files that are 1MB or smaller. The submission of these materials is not required; however, please note that a maximum of three files will be accepted per nominee. These files will be added to the end of your Case Study and will be labeled as "Appendix 1", "Appendix 2" or "Appendix 3." Finally, feel free to reference these images in the text of your Case Study by specifically referring to them as "Appendix 1", "Appendix 2" or "Appendix 3."

Currently Uploaded Appendices:
No appendices currently uploaded.