QA Tool Review: WAPT 5.0

Okay, I decided to get the last of the QA tool reviews out on the site so I don’t have to think about them any more. Here it is:

Description

WAPT is a performance/load testing tool developed by SoftLogica. It is a Windows-only tool designed to be very easy to use. A free 30-day trial is available at the company website that allows full functionality with the restriction of 20 virtual users.

Overview

Pros

Very easy to record and run tests
Decent reporting

Cons

Not Free
No scripting capability
Doesn’t do well recording AJAX calls

Pricing

Standard license - $350 with price per license decreasing with volume
- No distributed load generation
- Unlimited VUs
WAPT Pro - $700 includes 1 standard license and 2 load agents
- Additional Load Agents - $200
- Provides distributed load generation

Ease of Use

Ease of Installation

WAPT was super easy to install. It installs just as easily as any Windows program. The installer puts everything where it needs to be and you’re set to go.

Setting up a Test

Setting up a test is as simple as starting to record, then going to the sites you want to test. WAPT records all your activity. You then organize it into Initial, Main, and Final actions. The initial actions are executed once per VU per round. These would include things like logging in. The main actions are completed as many times as you tell it to in the virtual user profile. Final actions are completed after all iterations of the main actions.

Once the tests are recorded, you set the Test Volume information and run the test.

I docked WAPT 1 point in this category because the recorder does not do a very good job with AJAX applications. It did okay with Gmail but not so much with LDS Maps.

Running a Test

Running a test is a piece of cake. Once it’s all set up, click the run button and you’re on your way.

Technical Knowledge Needed

Little to no technical ability is needed, since there is no scripting or programming required. You can do most tests just by recording them and playing them back.

Platform

WAPT is Windows-only. Requirements:

Windows 98/Me/2000/XP/Vista (also works on Windows 7 RC)
Internet Explorer 5.5 or higher
At least 100 MB free hard drive space (for program + logs)

Recommended:

Pentium 4 or Athlon XP processor
512 MB+ RAM
Gigabit ethernet

Functionality

Recording a test

WAPT uses an in-application browser to record test cases. This is a great feature when recording https test cases. Most other tools use a proxy which is unable to detect https GET/POST data.

Unfortunately WAPT doesn’t do a great job recording AJAX application scenarios. Some AJAX requests are missed by the recorder, and thus would have to be created manually. With the Internet moving more and more toward rich applications, this will be a problem if they can’t get it resolved in future versions.

Scripting

WAPT does not support any kind of scripting.

Reporting

The reports can be saved as html files which includes some summary data and basic graphs. Results can also be saved to a CSV file if a more detailed data analysis is needed. Figure 1 and Figure 2 below show examples of the information that is available in the HTML reports. As you can see from Figure 2, the data is grouped together by time period. These groupings can be adjusted. The current setting groups them into 10 different segments. That number can be raised or lowered as desired. The list below is all the statistics that are calculated and included in the HTML reports.

Avg response time (also min/max per page)
Avg download time (also min/max per page)
Pages per second
Hits/sec
Sessions/sec
Total Kbytes sent and received
Received and sent kbits/sec
Received and sent kbits/sec/user
HTTP errors (including user-defined validation errors)
Socket errors
Timeouts
Active users
Total pages, hits, and sessions

[caption id=”attachment_85” align=”alignnone” width=”588” caption=”WAPT Report Graph (HTML)”][/caption]

[caption id=”attachment_86” align=”alignnone” width=”560” caption=”WAPT Report Table”][/caption]

Trend Reporting

WAPT’s graphs are great at showing the trend throughout the test. Each metric is measured for each time period and the graph shows the results live while the test is running. These results are also available in the CSV file exported.

Integration

The only external integration WAPT offers is the ability to export to CSV.

Data Functions

In WAPT this is possible, but not as smooth as with jMeter and others. It is done by creating a variable, selecting its function to be “Random from list” or “Ordered list”, and then loading the strings from a file. It this way, each parameter (e.g., street, address, city, etc.) needs its own file. Not the most efficient way. Other programs allow you to use a CSV with each column as its own variable.

Load Generation / Distributed Load Generation

WAPT does not support distributed load testing. WAPT Pro does, but the license for that costs twice what the regular license is. WAPT Pro includes the ability to generate load with 2 machines. More “load agents” as they are called cost $200. There is no limit on VUs, though, so your ability to create load is limited only by your hardware.

Server Monitoring

WAPT does not include any ability to perform server monitoring. WAPT Pro does.

Documentation

The documentation that comes with WAPT is great. Because the tool is so simple, it really doesn’t need a lot of documentation. WAPT is not feature-rich, so the little documentation that exists effectively covers all of its functionality.

User Interface

Fairly pretty UI. Bubbly icons look a little ‘90s, but it’s not bad.

Performance v. Stress Testing

WAPT supports both static-load performance testing and ramp-up stress testing. It also allows you to select whether you want the tests to run a certain number of times or just for a certain time frame. The screenshot below (Figure 3) shows the setup screen where you select how many VUs will be in the test, the ramp-up time (if there is any), and the final load. WAPT also provides an option for “Periodic” load, which alternates between two levels on a schedule you set.

[caption id=”attachment_87” align=”alignnone” width=”300” caption=”WAPT Load test setup”][/caption]

Bugs/Weirdness

The built-in report graphs are great, but they use both sides of the y-axis for different scales (Figure 4). There is no indication which line is associated with which scale. You have to turn items off and back on to be able to figure out what goes where.

[caption id=”attachment_88” align=”alignnone” width=”530” caption=”WAPT report graph”][/caption]

Project Info

History

SoftLogica is based out of Novosibirsk, Russia. Their website appears very out of date, with the most recent press release from 2007, and the newest newsletter from 2006.

Frequency of Releases

New major versions of WAPT are released on average every two years. The newest version (5.0) was released in May, 2007.