Description

The Grinder is a JavaTM load testing framework that makes it easy to run a distributed test using many load injector machines. It is freely available under a BSD-style open-source license. Below is a list of some if its key features.

  • Generic Approach Load test anything that has a Java API. This includes common cases such as HTTP web servers, SOAP and REST web services, and application servers (CORBA, RMI, JMS, EJBs), as well as custom protocols.
  • Flexible Scripting Tests are written in the powerful Jython scripting language.
  • Distributed Framework A graphical console allows multiple load injectors to be monitored and controlled, and provides centralised script editing and distribution.
  • Mature HTTP Support Automatic management of client connections and cookies. SSL. Proxy aware. Connection throttling. Sophisticated record and replay of the interaction between a browser and a web site.

Overview

Pros

  • Jython scripting makes tests very customizable

Cons

  • Scripts can be complicated to write and maintain
  • Fairly solid technical ability required
  • Proxy recorder does not record HTTPS traffic

Pricing

The Grinder is free and open-source. BSD-style open source license.

Ease of Installation

Installing The Grinder is a pretty complicated process compared to the ease of other tools. First go to The Grinder’s SourceForge page and download the current version. Then go to the “Getting Started” page and follow the instructions for creating the startup scripts and property files required. (Note: instead of creating a new grinder.properties file from scratch, I recommend copying the grinder.properties file from the examples directory. It gives a well commented base to start from.)

Once you’ve got all the scripts created, test it out by running startGrinderConsole.cmd and startGrinderAgent.cmd to make sure that the Grinder Agent properly connects to the Console to receive commands. If the “Start worker processes” and “Reset worker processes” buttons are enabled, then the communication is successful and you will be able to run tests.

Setting up a Test

The best way to set up a Grinder test is first to record a script, then adjust it if necessary. First start up the proxy on your computer by running startGrinderProxy.cmd (you should have created this file during the installation). Tell your favorite web browser to direct traffic to localhost:8001 (unless you changed the proxy port settings). Unless you changed the commands in startGrinderProxy.cmd, the proxy will create a file called “grinder.py”. When you’re done recording your actions, stop the proxy and change your browser settings back to normal. You can open up the newly created Jython script and edit it if you want, or you can run it as is.

Next you have to change your grinder.properties file to determine how many worker processes and threads/process you want to run. Set grinder.processes and grinder.threads to dictate how many processes you want and how many threads per process. Leave grinder.runs at 0 as long as you will be using the console. Be sure to change grinder.script to point to the script file you want to run. Once you have that set up, you are ready to run the test.

Running a Test

Once the test script has been recorded it is easy to run. Just modify the grinder.script property in grinder.properties to point to the correct *.py file. Then start the Grinder Console and an Agent process. Then click the “start the worker processes” button and your test will start running. When you want to stop click “stop collecting statistics” and then “Reset the worker processes.” That’s it.

After the test runs, if you decide everything went well enough that you want to save the results, click “File” à “Save results…” to save the tab-delimited results table. By default it is saved as a .data file. You’ll probably need to associate Excel with that kind of file, unless you have another preferred tab-delimited file reader.

Technical Knowledge Needed

Just to get the program set up doesn’t require a whole lot of technical ability, just the ability to follow instructions. The creation of a script using the proxy is also very simple. However, if you want to start creating your scripts from scratch or editing the recorded script, then you’re going to need some technical prowess.

Platforms

The Grinder is a Java application, so it will run anywhere Java runs. The Grinder 3 requires Java 1.4 or greater.

  • Windows
  • Mac
  • Linux

Recording a test

The Grinder provides a proxy tool that can be used to record tests. The proxy is used to create a Grinder script that can then be run from an agent. The proxy records every request that is sent out, and includes a timer to time the delays between requests. This is great for accurately simulating a user’s interaction with the system.

Scripting

The creation of tests is done entirely in Jython, the JVM implementation of the Python programming language. The recorder creates a Jython script that can be modified to fit exactly what you want. You can also create scripts from scratch, but the recommended method is to start from a recorded script.

Reporting

The built-in reporting consists of a scrolling graph of the 25 most recent Tests per Second (TPS) values for each test being run. There is also a table of data for each test. This is the same data that is available to be saved as a tab-delimited results file. The data reported is listed below:

  • Test #
  • Description
  • Successful tests
  • Errors
  • Mean time
  • Mean time Standard deviation
  • TPS (tests per second)
  • Peak TPS
  • Mean response length
  • Response bytes per second
  • Response errors
  • Mean time to resolve host
  • Mean time to establish connection
  • Mean time to first byte

Trend Reporting

The Grinder shows a graph during the test run that shows the tests per second for each of the tests being run for the past 10 or so sampling periods. That is the extent of trend reporting. All other numbers are reported in aggregate for the entire test run. The only numbers available in the CSV report are aggregate.

Integration

The Grinder exports all test results as a CSV file. That is the extent of its integration with anything.

Data Functions

Since the tests are all run using Jython, anything that can be done with Jython can be done in the tests.

Load Generation / Distributed Load Generation

Although I have not tried it, the ability to generate a distributed load seems to be very straightforward. All you need is to start an agent process on another computer and modify the grinder.consoleHost property in the grinder.properties file. That agent will then be ready to receive commands from the console and report back.

Application performance when generating a load is very good. I was able to run 400 threads (20 worker processes with 20 threads each) without putting much of a dent in system performance at all. Starting up all the processes maxed out my CPU for about 15 seconds, but after all the processes were started the CPU ran at about 25% with spikes up to 50%. The 400 threads consumed about 512 MB of RAM.

Documentation

The Grinder’s documentation is good, but slightly fragmented. The User Guide on the home page is fairly complete, but it is not well organized. The Javadoc seems to be complete and well commented, although I didn’t use it very extensively.  There seems to be an adequate amount of community documentation and help as well. It is slightly more difficult to find because “Grinder” is a fairly common word, unlike WAPT or JMeter.

User Interface

The UI is a standard Java Swing interface. Fairly ugly, but it gets the job done.

Performance v. Stress Testing

The Grinder allows for both steady-load performance testing and ramp-up stress testing. Several properties in the grinder.properties file allow you to set the preferences for stress or performance testing. Check out the page for the properties file (link) to see all the settings you can adjust.

Bugs/Weirdness

  • The Grinder is shipped with a version of Jython but does not package the Jython distribution of the standard Python library. If you want to use the standard library, or if you want to use a different version of Jython, obtain and install Jython and tell The Grinder where you installed it. [taken from the Grinder user manual]

Project Info

History

The Grinder was originally developed for the book Professional Java 2 Enterprise Edition with BEA WebLogic Server by Paco Gómez and Peter Zadrozny. Philip Aston took ownership of the code, reworked it to create The Grinder 2, and shortly after began work on The Grinder 3. The Grinder 3 provides many new features, the most significant of which is Jython scripting. Philip continues to enhance and maintain The Grinder.

In 2003, Peter, Philip and Ted Osborne published the book J2EE Performance Testing which makes extensive use of The Grinder 2.

Frequency of Releases

Limited information is available on the frequency of releases for The Grinder. There are three versions available on SourceForge that were released in 2008 & 2009. The most recent version (3.2) was released in February, 2009.

Ohloh.net reports that The Grinder is experiencing decreasing year-over-year development activity.

Active Forums

Official Forums

Unofficial Forums

Company URL

http://grinder.sourceforge.net/