|
|
Performance Testing Ensures A Reliable,
Predictable Experience |
A Review of a Performance Analysis
Conducted By RTTS
A market research firm has developed an
online survey application that is in the
final development stages. The purpose of the
test suite we conducted was to help
understand the performance characteristics
of the site and the application they created
for users to enter surveys. At the
conclusion of the test, the firm's president
stated, "Load testing was particularly
critical prior to rolling this application
out for our client. After RTTS' performance
analysis, we realized that their services
will be a good fit for our future testing
needs."
|
|
Software Development/Market Research |
How does a development company ensure the
performance accuracy of the application and
will it meet the service level agreements it
has with its clients?
|
|
Outsourced
performance testing conducted by RTTS to
detect scripting or networking errors in the
development stage and continually monitoring
performance during implementation. |
• Using the automated testing tools we have available, RTTS' experts can
evaluate and ensure the scalability and performance of Web applications.
• Using a Fixed Price Testing program, you will know the cost and deliverables
up-front
• Outsourcing the test analysis and performance testing will provide the
development team with the time to concentrate its efforts solely on their
mission.
|
• The confidence that your applications operate at peak levels enables the
development firm to keep satisfied customers and business partners.
• Automated testing eliminates time-consuming repetitive data entry.
• Data-driven tests ensure comprehensive validation of complex rate schedules.
• Test cases easily and frequently revised to absorb continuous site changes.
• As testing experts, RTTS can be a single point of contact amidst a multitude
of technologies.
• The time-to-market is greatly diminished when outsourcing performance testing
as a "one-time only" task.
|
Application Overview
The application has only been production ready for a short amount of time and
has not been accessed by live customers as yet. Web users would access the
application, which resides on an internal server, in response to a direct
electronic messaging campaign conducted by the client. These e-mails would be
sent in packets of 1,000 messages, limiting the anticipated number of users
accessing the site at any one given time.
Transaction Scripts
One business transaction was scripted for these tests. The complete "survey"
business transaction was divided and timed on a page-by-page basis. Eight
individual pages were accessed during the script.
Virtual Users
User-level tests were executed ranging from 1 to 75 concurrent users over a
predefined period of time. Each virtual user accessing the site performed data
entry into a mock up survey. At the beginning of each test the users were
started one at a time every 2 seconds to keep the test system from saturating
and to obtain a steady state of response as quickly as possible. After each user
completed their first transaction, subsequent transactions were not paced (no
delay before starting the next transaction). Each test was run for 15 minutes to
gather enough data to be statistically relevant
Results
The following results were obtained and extracted from the data collected:
• Page Load Times - There was a large drop in performance when the load was
increased from 10 to 15 users. This equated to a 45% decrease in performance as
compared to a single user.
• Page Volume - A linear decrease in accessed pages. This is the result of the
performance decrease observed by the page load time analysis.
• CPU Utilization - The system approached saturation at 15 concurrent users.
 
As tested, the application scaled to 10 concurrent users before the target
system CPU saturated and resulted in a decrease in performance and page volume.
Recommended Course of Action
Execute additional tests to analyze the behavior of CGI environment and
corresponding Perl scripts. Historically, CGI applications tend to be resource
intensive due to the "forking" or creation of new processes for each CGI-related
request. Optimize the application by performing one or more of the following
tasks:
• Employ more detailed performance monitor counters to drill down on the source of
the high CPU utilization,
• Analyze the access methods for the application's data storage to ascertain the
possibility that data access is being serialized via database locking or file
system limitations,
• Evaluate the manner in which the application is handling simultaneous requests.
These tasks should be executed in an iterative manner until response times and
any other performance requirements meet expectations. It is recommended that
modifications to the application and environment be made one at a time in order
to determine the sources influencing the performance of the application. |
|
|
|
|
|
| |
