Case Study

Background

A worldwide beverage corporation depends on over 75 internal and external applications to perform critical business processes by multiple teams.

Because the Windows XP operating system is nearing End-of-Life (EOL), this corporation is preparing to upgrade to the Windows 7 operating system.

Along with upgrading to Windows 7, there is an internal team that customizes the platform to include specific software before rolling it out to the end-user.

The team continuously releases new builds of the Windows 7 operating system that contains patches and upgrades for existing software.

The goal of the software quality effort is to verify each application against each new build of an operating system.

Challenges

How can a software quality effort verify that critical applications do not break when a new build of the operating system is released?

Strategy

Implement regression testing using automation to verify each application for each new operating system build.

Solution

• RTTS used QuickTest Professional 10.0 to create automated test scripts for each application being used.
• Approximately 150 scripts were created that covered different transactions for each application.
• Quality Center 10 was also used with QTP to create test sets that can run each QTP script remotely on different machines at a scheduled time.

During the script development cycle, a meeting was set up with each application team to manually test their application once on a Windows 7 machine that had UAC (user account control) enabled.

This was done to ensure that the end-users won’t have any issues with their application due to UAC, which requires administrative rights to proceed. Each application took about 15 – 30 minutes to manually test. 

Automated scripts were then created to test the different functionalities of each application. Scripts were reviewed and verified by the each application team manager. On average, script execution took approximately 1 – 2 minutes to complete. 

Each script was then run on a weekly basis to check if any of the applications broke due to changes made to the operating system on a development machine.

Without automated scripts, it would have taken approximately 20 – 40 hours to manually test 75 applications. By using automated scripts, a full test cycle was cut down to 1 – 3 hours.

Additional scripts were also created to test whether IE8 was configured correctly in terms of domains within the IE8 trusted site setting. These tests were performed by checking Windows registry keys. This script provided the development team with a table that contained the path and name of the registry key, the expected and actual values, and when a test failed, whether or not it failed due to value difference or the key not existing.

Benefits

• Reduce overall time needed to test applications by a factor greater than 10.
• Schedule tests to be run at anytime during the day.
• Avoid the reliability/reproducibility issues with manual testing.
• Quality Assurance team can easily validate expected data with actual data.
• Using regression testing leads to high confidence that applications will not break when patches and updates are introduced to the operating system.