RTTS: Automated Performance Testing Case Study with Rational

RTTS' Experts Put IBM Rational Performance Tester to Work

A Review of a Performance Testing Engagement Conducted By RTTS

Industry/Market

RTTS' client is a global travel and entertainment company.

Challenge

The application-under- test was a new release designed with better navigation, more attractive marketing, enhanced searching and purchasing capabilities. Accessible over the Internet, the current user hit rates were in the thousands hourly. The new site was scheduled to launch in one week and the team was experiencing unusually high response times under test conditions along with inconsistent test executions, RTTS was asked to work with the client team to analyze and validate the test results within the short timeframe.

The following goals were established:

		Identify weaknesses in the current testing processes, automation practices, and methodology and provide correction action
		Review and modify the existing test scripts to improve reliability and execution
		Optimize the automated testing tool, enabling it to emulate the required number of users in the test environment
		Assist in test execution and analysis

Solution

The client was using IBM Rational Performance Tester to verify the production readiness and scalability of the new release site. An ideal choice for this challenge, this product determines whether an unreleased application, when under multi-user loads, performs as needed against specific response criteria.

Test Highlights

Working in partnership with the client's on-site test team, RTTS was able to meet these goals within 3 days, delivering rapid results and providing the required confidence to meet the release deadline.

	- Day One: RTTS optimized the test tool driver configuration.
	- Day Two: RTTS modified the automated test scripts for accurate duration, page and URL timings. RTTS also added test management code for more accurate control over test execution with the ability to ramp up and down the virtual users, as needed.
	- Day Three: Using the modified test scripts, RTTS provided repeatable tests so that the test infrastructure could be analyzed to determine the cause of the poor performance. With the ability to run repeatable, well-defined tests, the client found major issues within their database and web servers.

Quantifiable Results

With a day to spare, the client was able to launch the new site with assurance that it would perform as expected, and the results showed it. The client reported the following record-breaking results to RTTS:

		Server response time in production averaged 2 seconds
		Average dynamic pages served ~60,000 pages per hour at an average of 1.7 seconds
		Web traffic reached over 80,000 unique visitors, 10% above the last all-time record
		At peak load, the site performance averaged under 2 seconds serving dynamic pages and forms
		Over 80,000 unique visitors were on the site in one 24-hour period, breaking previous records
		During one peak hour 10,00 unique visitors and 3,000 repeat visitors were served 100,000+ dynamic pages at an average of 1.7 seconds

RTTS was able to assist their client in quickly deploying a quality application by leveraging the features found in the Rational product suite. Furthermore, by solving problems in the test lab, the costs for diagnosing issues and tuning application performance are the lowest. The client was able to minimize expenses and provide a positive end-user experience.

To learn more about our performance testing services, click here.