PEMCO is a preferred-risk insurer that offers Auto, Home, Boat, Life, Mariner and Umbrella Insurance to Washington state residents. The company has been catering to insurance needs since 1949 and efficiently serves more than 500,000 customers. Started as School Employees Credit Union by Robert J.Handy with a small capital of $5, today PEMCO is a vital player in the state of Washington. PEMCO has been recognized by J.D Power and Associates for its commitment to an outstanding customer service experience for three consecutive years. Currently PEMCO is working toward enhancing customer experience and focusing on several new initiatives to create a positive difference for the customer.

Since 1982, PEMCO had used a legacy PMS system for selling new policies and serving its existing customers. However, in 2003 PEMCO decided to move to a web-based system for operational efficiency, cost efficiency, scalability and speed to market. PEMCOs IT wing focused on CSCs Exceed insurance application, which is a one-stop shop for Personal Insurance Business and decided to customize it to its needs.

PEMCO Test Delivery Center (PTDC) was set up with a vision to provide high Quality Assurance services across all projects. PTDC is based on an onsite - offshore model with Cognizant Technology Solutions (CTS) as its strategic partner. CTSs extensive testing experience, deep industry knowledge, partnerships with major testing tool vendors, and expertise in testing tools that provides accelerated testing at reduced costs were the deciding factors for the collaboration. 

PTDC has developed sophisticated testing tools, processes and proven testing methodologies. It provides a broad and expanding range of QA services like Functional Testing, Conversion Testing, Regression Testing, User Acceptance Testing, and Production Support Testing. It has a pool of highly skilled professionals who ensure total quality by uncovering bugs during early stages of the project. PTDC optimizes the test cases for every cycle to reduce the testing cycle time for each release. Core-Flex Resource model is implemented in PTDC to support fluctuating resource requirements. 

However, PTDC has faced many challenges like validating data conversion from legacy system to Exceed, data secrecy, adhering to Agile principles in a distributed environment, large number of scenarios for complex business functionalities, strict release schedules, knowledge transition and environmental availability. 
But PTDC responded to these challenges by effectively leveraging technologies and tools like Automated Conversion Testing, Distributed Agile Testing, Orthogonal Array Testing, Test Automation, Online Knowledge Management and Automatic Environment Tracker. PTDC is helping PEMCO to achieve improved performance and quality, reduced cost of ownership, reduced time to market and lower operational costs. 

PEMCO is now moving toward an e-Commerce Platform, where PTDC is poised to achieve success with its standardized test processes and best practices.

The aim of the PTDC team was to provide complete and efficient QA services with an Onsite Offshore model to achieve cost benefits. However, the change in the core business processes - due to legacy modernization, Agile development, Data Secrecy, and Infrastructural Requirements -- were posing tremendous challenges. 

The PTDC team used the latest cutting-edge technology and techniques to overcome these challenges. It ranged from simple tools that provide finer control over the environment to complex automation tools for managing the most complex aspects.

-Conversion Testing Reusable Automation Tool (ConTRAT): This tool was developed and used in the Conversion Testing of the core business change from legacy system to Exceed. The tool automates the validation of the converted data. Compared to the manual process it had given a cost savings of 96%.

-Orthogonal Array Testing (OAT): OAT is a statistical technique that helps in arriving at an optimized number of test cases from an exhaustive set of possible combinations of test parameters. The OAT tool was able to optimize the test cases to 1,450 from a possible 18,000 combinations for a specific functional area. This translates to 91.9% of cost-savings. 

-Modified Condition and Decision Coverage (MC/DC): MC/DC is a testing technique to test complex logical conditions. It allows identifying a specific combination of conditions that affects the outcome of a business function, leaving the duplications. This technique was used in one of the critical business functions that allowed the underwriters to make critical decisions. This scenario demanded best coverage with all possible testing conditions. MC/DC was used to provide the required coverage with reduced number of test cases. 81% effort reduction and 78% cost savings were achieved compared to the traditional testing methods. 

-Totally Extensible and Reusable Automation Framework (TERA): TERA was one of the key elements of the test strategy to reduce effort and cycle time. TERA was a robust reusable Keyword Driven Automation Framework developed for the demands of Agile methodologies. TERAs advance scripting methodology reduced 31% scripting time compared to traditional automation scripting. It assisted in reduction of test execution time and cost with assurance of 100% quality. 

-Distributed Agile: Unique and first of its kind, Distributed Agile was the key for the successful execution of QA Services. The Onsite Offshore model posed a challenge to the Agile Development, violating some of the Agile principles. PTDC used an Online Story Board tool that enabled the Offshore testing team to interact with the Onsite team according to the Agile principles. This allowed PTDC to realize the benefits of both Onsite-Offshore model and Agile development like increase in savings, reduced testing time and faster release cycles. (For more details, please refer to the Originality section).

-Online Knowledge Management Portal: One of the challenges faced by PTDC was effective Knowledge Transition in the Offshore Onsite model where the teams are in different geographical locations. PTDC implemented an online repository of all project documents, video and audio training materials, and a wealth of scenarios designed over a period of time. This facilitated effective online training not only for PTDC resources but also for the Sales Insurance Agents.

-Environment Downtime Tracker: The 24X7 availability of the test regions to the PTDC team was one of the critical success factors. Tracking the availability of multiple Testing regions manually was a daunting task. To manage this, an automated tracker tool was developed to monitor the connectivity of the test regions on a 24X7 basis. The tool raised mail alerts upon any unplanned downtime to all the stake holders. Daily impact reports helped the PTDC team to escalate and address the loss of productivity.

Yes

Yes

The legacy system conversion was an organization-wide initiative and it required effective coordination and communication between all its stakeholders. Agile testing demanded an attitude change within the resources and they successfully converted numerous challenges into opportunities. A few of the advantages and opportunities are listed below: 

Reduced Cost of Ownership: 
-The ever-changing resource requirements were addressed with a Core-Flex resourcing model. A Core team was utilized to its full capacity and a Flex team was managed for any ad-hoc requirements. The Capacity Planning Tool helped in predicting the number of flex resources for a specific period. 
-Reduced average costs through an optimum Onsite Offshore ratio. 
-Test suites were continuously optimized by eliminating test cases (for example, cases that are consistently successful). It led to a reduction in cycle time and increased test execution productivity, which resulted in a 92% reduction in effort. 
-Testing of defect fixes was automated to reduce the effort during Regression Testing. 
-Insertion of test data to the Test Databases was automated and it reduced manual effort. 
-Risk-based testing enabled the testing team to focus on the areas of the application affected by the change/defect fix delivered in the test environment. 
-PTDC was able to bring considerable cost savings by setting common testing environments for different projects, clubbing smaller builds for one-time regression, and sharing tools and licenses between different teams. 


Reduced Time to Market: 
-Continuous optimization of test suites ensured reduced testing cycles without sacrificing quality. 
-Maximizing Test Automation (75%) helped in reducing the testing cycle times. 
-Due to the continuous availability of test environments, project schedules could be maintained. 


Improved Quality: 
-Proven techniques like OAT and MC/DC helped to achieve increased test coverage with an optimum number of test cases. 
-Agile testing ensured early detection of bugs, and these were fixed and tested in subsequent releases. Zero defect leakages were found in most of the releases. 
-Consistent methodologies, processes, tools and metrics enabled us to maintain the high quality. 
-Increased quality ensured higher customer satisfaction (measured through regular surveys). 


Risks Mitigation: 
-Standardized operational and management processes to reduce people dependency. 
-A pool of trained resources to cater to the ad hoc work requests. 


Opportunities: 
-Agile testing demanded a total change in the processes and methodologies as followed in a conventional testing approach. Resources were trained onsite in an Agile environment and then relocated to Offshore for project execution. This gave resources a great opportunity to learn and be a part of Agile testing. 
-The Onsite - Offshore model required the team to develop an Online Knowledge Management system to do an effective knowledge transfer across locations. This gave the team an opportunity to understand the overall domain. 
-New resources were trained through a Boot Camp Training portal in which audio and video materials enabled them to learn the domain and application quickly. An evaluation was done before resources were put on the projects. 
-Scenarios like PEMCO changing their banking partner offered PTDC resources additional exposure.

The modernization program benefited the organization as a whole: 

-PEMCO was able to release products to market in 3 weeks instead of 6 weeks. One of the key factors for the reduced time-to-market was the reduction in cycle time by Virtualization. 

-Due to the Agile test environment, the development team received immediate feedback on the build. 

-75% of the manual Test Cases were automated. The Regression test cases of Auto Line of Business and Property Line of Business were automated as a part of this project to speed up the process of validating the build and reduce manual effort. 

-Cycle time reduction for Regression Test Suite to 25 minutes was achieved through Virtualization. (For details refer to the Originality Section.) 

-RateWhiz was able to reduce the time taken to validate premiums from 500 minutes to 2 minutes for a batch run. (For details refer to the Originality Section.) 

-PEMCO realized cost benefits in the year 2007-08 because PTDC achieved cost reduction in Project Spending (20%), Performance Testing (13%) and Support Activities (22%). 


Fellow members of the Computer Sciences Corporation (CSC) customer community voted PEMCO Insurance as an industry leader for its work in implementing quality management best practices to solve a particular business need in an innovative way. PEMCO received the Quality Management Leadership Award -- the first of its kind at CSCs annual Insurance Industry Connect Conference in Orlando on Sept. 30, 2008. One PEMCO Direct Insurance agent wrote, Now that we are into a couple of years and a lot of refinements with the front end, I think it is the best project implemented here in my 22 years at PEMCO. Just thought you should know.

Below are some additional Quotes:

"This message is sent on behalf of the PEMCO executive team. We did it! We're growing again! PEMCO reached its final Go to Market milestone at the end of first quarter 2008 by blowing past + 1 net growth in policies. We grew by 705 over the 2007 year-end total of 349,492 for all in-force policies (auto, home, all other P&C lines, and life). We came out of first quarter with growth in each P&C line and life"-By Executive Team.

"This is excellent news. I am impressed with the gain in productivity and have forwarded to IT Director as a great example of process improvement that directly affects cost and quality. Please extend my thanks to all involved"-By QA Manager.

"The QA of this project to me is inspirational. The QA of this team has an offshore which is not familiar to agile testing. They are not even here with us. QA brings in best collaborative ideas. You guys really helped us to think out of the box"-By e-Commerce Project Manager.

"I have run into many challenges & difficulties, but the one bright spot that always kept me in great spirits is my QA team, I can go out and talk everybody with confidence that this team can deliver, knows how to do things RIGHT. You guys are our ONLY SUNSHINE!!!"-By IT Director PEMCO.

"I am really excited about the QAs breadth of Insurance Knowledge and their willingness to suggest different ways of looking at the same problem and providing better solutions to solve it in a quicker way"-By Chief of Research, PEMCO.

Distributed Agile Testing: PTDC adopted the Agile testing methodology distributed across PEMCO locations without any part being off-shored. As the process matured PTDC adopted Agile testing methodology in an Onsite-Offshore model, a major component of which was carried out at offshore by the virtual onsite team. It was built using efficient testing methodologies and tools that enabled the virtual onsite team to communicate efficiently via the Online Story Board Tool bringing all stakeholders together on a real-time information sharing platform. This tool replaced the traditional way of maintaining post-it notes by online stories and testers could trace the background through archives. 

Some of the benefits realized were: 
-Reduced cost by 25% as teams could be distributed across geographic locations. 
-100% Automation of regression to support frequent releases. 
-Capitalize on time zones, enabling 24 X 5 testing window. 
-Accommodate dynamic market changes. 
-Testing small frequent releases and end-user reviews. 
-Management also used the Online Story Board to communicate. 
-Development and testing happened in parallel throughout the project, avoiding a separate phase for testing. Defects were identified at an earlier phase of development, which avoided defect leakage. 


Virtualization: To realize the management vision of a faster time to market, the PTDC team set up a first of its kind Virtualization lab to automate the Regression Test Suite. The virtualization server was set up to simulate multiple operating system instances to run concurrently on a single computer. It provided seamless transitions between different operating systems on a single machine, which reduced desktop footprint and hardware expenditure. The pilot phase of the virtualization automation execution of regression test suite was carried out using WinRunner. 

Benefits: 
-67% reduction in execution time. 
-Availability of automation resources for other testing activities. 
-More rounds of regression testing possible. 
-Virtual Lab setup could also be utilized for other testing needs, like browser compatibility testing, etc. 


RateWhiz: To validate the Insurance Premium calculated by the System, PTDC developed an Excel-based Rate Order Calculation tool that could calculate the Insurance Premium on providing policy details as input. Manually verifying the Premiums was a time-consuming process. RateWhiz was developed to completely automate this process. It extracted policy details from the application database, fed it to the ROC tool and compared the ROC result against the System calculated value. It processed 3,500 insurance policies per hour against the manual effort of 25 policies per day. It also produced a report highlighting the discrepancies. 

The key benefits derived from the tool were:
-The manual effort of verification was greatly reduced. 
-Renewal premium discrepancies were identified and fixed well ahead of sending bills to the customers. This substantially reduced the complaints against premium inaccuracies. 


R-AIR (Remote Automation scripts Invocation and Report generation) Using this tool Automation Scripts in a system were launched by sending an e-mail. This minimized manual intervention. This is a secure tool and only authentic users could access and use this. 

Benefits: 
-Multiple requests were processed through Execution Queue. 
-Removed dependency on the availability of the automation tester. 


-Quick Re factor: The frequent change in the application was a challenging aspect of Extreme programming. This led to more rework in the automation scripts. The Quick Re Factor tool provided a user-friendly interface with various options to make script changes and update the overall test suite in minutes. 

Benefits
-Changes that would take days to do manually were done in minutes by the tool. 
-Even a person not familiar with the scripts could modify and maintain the Automation Test Suite.

Due to the huge complexity of the legacy transformation and vast application landscape, the risks and problems were identified earlier, and the PTDC team worked to address the challenges coming their way. PTDC wanted to have a proven process with an excellent Quality Assurance (QA) methodology that could meet all the QA needs in a short span of time. 

-Test environment availability was a challenging part at PEMCO due to the network and server maintenance issues. As automation and manual testing were happening day and night, any environment issues affected the testing, which in turn impacted the project timelines. 

-Due to the number of projects and ad-hoc requests coming from business, it was very difficult to do the capacity planning for PTDC. Also the lack of code quality of one project impacted the dependent projects timelines. 

-The year on year cost implications put pressure on the project schedule and timeline. 

-There was a challenge for the team to be scalable in size and skill set whenever there was a requirement. 

-Retaining key resources. 

-Managing cultural differences.

-To provide the right tools and techniques that preserve or improve the quality characteristics of the Policy Admin System. 

-The need to plan and execute effectively without bringing any imbalance in PEMCOs business continuity process. 

-Many conversion testing projects have failed because of the lack of a mature conversion-testing mechanism to ease the complexity and technical challenges. To be successful in a program of this magnitude, the challenge was to develop a flexible conversion-testing mechanism supported by the use of automated tools and proven cost-effective strategies. 

-To develop a mechanism for identifying risks at regular intervals, right from the inception.

-To implement an effective Offshore-onsite engagement model.

-Managing morale of internal and vendor employees. 

-Selling of best practices developed within PTDC to other LOBs.

In 2004, when the engagement started it was proposed to be an onsite-offshore model. The reasons cited were to provide 24x7 support for testing activities and to provide a cost-efficient solution. 

Initially there were lots of obstacles in setting up network connectivity between the development team and the PTDC. Providing the PTDC team the access to test systems in the development team location might lead to security issues for the production system. Later this was addressed with the help of a proper encryption model recommended by the security department, and all the access from PTDC to the development team was routed through PEMCO network. 

In addition to the QA activities, PTDC also provided Business Intelligence support (Reporting Support) for PEMCO. One of the responsibilities was to support the Production data correction team by providing reports. These reports were created from the Production system daily . When this role was originally proposed, there were obstacles for providing Production access to the offshore team due to security policy violation. After having discussions with the Security Department, the team was set up at an onsite location within the PEMCO network to perform this activity.

Exceeded

There were nearly 15 major projects implemented for PEMCOs Policy processing application. Based on the learning from various projects, quality was given highest priority as compared to cost and time. Every project that gets implemented is thoroughly QA stamped by PTDC. Only defect-free projects, which met QA exit criteria and business needs, were approved for Production. 


Overall Testing Cycle Time Reduction: 
With the increase in the number of projects from business, PTDC performed exercises with frequent process tuning and introduced Lean Six sigma and Risk-based testing. The Regression test suite kept on increasing with the projects, and it required more areas to be tested, thereby increasing the overall testing cycle time. PTDC came up with the Risk-based testing approach, which focused on high risk areas to be tested rather than the entire test suite. This reduced the overall cycle time and increased the frequency of projects and changes implemented. 


Same Test Coverage with cost-effort reduction by OAT and MC/DC: 
OAT (Orthogonal Array Testing) Methodology and MC/DC (Modified Condition and Decision Coverage) technique have been implemented as best practices to validate the rating engine and underwriting rules for PEMCOs EXCEED application. Products and changes are tested with reduced cycle time and improved quality. Business users got a quality product with a quick turn-around time and made PEMCO very competitive among all Insurance providers. High quality System maintenance tools developed and tested at PEMCO were shared with a partner company like CSC so that it would reach other EXCEED customers in the global community.

In September 2004, PTDC started testing of the EXCEED application, which was the latest offering in the Insurance processing system. The rating engine was the core module, which required thorough testing for customers to get the correct premium for their policies. The Rate Order Calculation tool simulated the EXCEED rate engine, and 10,000+ combinations of data were verified using the automation tools. Within a period of 6 months PEMCO business users started using this tool for quick premium validations. Within a year the Property Line of business also implemented this tool successfully.

In 2006 PEMCO decided to move the new development projects from traditional SDLC to Agile methodology. As a result, a new front end for EXCEED called PEMCO Sales & Service was developed and implemented phase by phase within a year for all lines of business. This methodology was to ensure continuous project delivery for business within a short period of time. It was appreciated by the stakeholders and contributed to tremendous growth within a short span of time.

A Comparative rater tool was built on top of the PEMCO Sales & Service and was used by Community Agents to compare the rate against multiple Insurance providers. PTDC introduced new features in the ROC tool and named it RateWhiz in December 2008 to cater to business needs. It helped in monitoring everyday renewal premium in Production and validates it against the ROC tool premium automatically so that customers are charged accurately.

All these new developments resulted in a significant increase in the number of policies that PEMCO could issue and in turn increased the business for them. The new infrastructure and tools improved the stability and reliability of applications for PEMCO, and they were able to support initiatives like Go To Market, e-commerce strategy and Voice of Customer.