Traditionally, setting up a load testing environment and creating even simple scenarios was quite difficult. But since I discovered the open-source tool Taurus, my opinion has changed a lot. Taurus provides an abstraction over a lot of common tools for load and/or performance testing. For example, some of the better known tools are JMeter, and Scala for Gatling.

In my opinion the really nice thing is that it provides unified and simple specification of test scenarios by using a common YAML dialect. At the same time, it is still possible to specify tests in the native formats of the tools, so you can still use HTML for Selenium, XML/JMX for JMeter, and Scala for Gatling.

Especially this mix of limited, unified and specialized setup allows me to still use the flexibility of JMeter and at the same time benefit from the performance of Gatling.

Another advantage of Taurus is, that it does not only unify the configuration and specification of tests, but also merges the results of the underlying test tools during a test run. So, the time-consuming process of manually combining log files and test artifacts after a test run with self-written scripts, templates etc for further processing is no longer necessary.

Installation and Upgrading

To install Taurus locally on Linux, follow these steps:

1. Install Python and pip (if not already installed):

   sudo apt update
   sudo apt install python3 python3-pip
   

2. Installing Taurus via pip:

   pip3 install --user bzt
   

3. Check the installation:

   bzt --version
   

Ready! Taurus is now installed and ready to use. I can use it by the cli command bzt in combination with a yaml file test specification, to stress what ever system. Optionally it can be installed in a virtualenv, so that you can run it in a dedicated environment. And if you are currious now "What's about the tools used in the tests", all used tools will be installed automatically during the first run by Taurus. It is enough to specify those tools in your/my test plan and then everything happens automatically under the hood by Taurus - hopefully 🤞 You will read more about it in the next section.

Background Load with Taurus

I must admit, that Gatling wasn't in my scope, before I started to work with Taurus. But since I work with this tool continuously, I really like the taste of this tool. One mayor reason is that Gatling is based on Scala and with that technological background it performs somehow awesome and ressource friendly. If I compare that to a similar setup with JMeter, it is my tools of choise, when I need to establish some kinde of background noice. And beside that there is an additional benefit for me. I can run a lot a) more parallel stuff or b) more complex test scenarios on the same hardware, due to the combination of different tools by purpose, to stress my system under test.

The following yaml file contains a very simple test plan, which will stress the domain https://pdepend.org with a solid background load for 2:30 minutes. While the total runtime of 2:30m consists of 30 seconds ramp-up and 2 minutes peak, with concurrent requests in 40 threads.

execution:
- executor: gatling
  concurrency: 40
  ramp-up: 30s
  hold-for: 2m
  scenario: pdepend

scenarios:
  pdepend:
    requests:
    - ${url}
    data-sources:
    - path: gatling-urls.csv
      delimiter: ','
      quoted: false
      encoding: "utf-8"
      loop: true
      variable-names: label,url
      random-order: true

To define the actual endpoints used in this test, I used the following CSV file, which is in the same directory as the test.

label,url
home,https://pdepend.org/
docs,https://pdepend.org/documentation/getting-started.html
screenshots,https://pdepend.org/screenshots.html
download,https://pdepend.org/download/index.html
support,https://pdepend.org/support.html

This test can now be started using:

/opt/taurus/bin/bzt 01-gatling.yaml

And now the magic mentioned happens. Because I used executor: gatling in the test plan, Taurus starts installing Gatling in the local environment before the first test run.

What was a real eye-opener for me when I first used Taurus was the real-time overview displayed on my shell. While my test plan was running, this view showed not only progress, but also various live metrics, like response times, throughput (r/s), and error rates per URL. The compact, structured output in this view clearly shows to me that an experienced QA/performance experts was involved in the implementation of this view. Beside that the shell view already allows to identify bottlenecks or anomalies. As a bonus, I get a clear result summary, including statistics on latency, throughput, and success rate, at the end of the test run. The following figure shows the real-time overview during the execution of the test plan.

That's it. With Taurus, I will got quickly and easily to create a test that generates background load or load on the web server. With this basic setup, more detailed tests can now be implemented to check the behavior of critical application functions under load.

Oh, what I almost forgot. After each test run, all elements used in the test (the plan itself, generated files, fixtures, logs, the tested files, and more) are archived in a subdirectory with a corresponding timestamp.

With this stored data and additional metrics recorded during test run, I can easyly setup a database that tracks the qualitative development of the software over time. With such a history, it is possible for me to build my own dashboards and use the data to visualize the deterioration or improvement of your software over time. At the same time, it is also possible to automatically detect recurring problems very early.

In the next post, we will expand the test plan a little and, in addition to the background noise, also put a function or workflow under load so that we can see whether a business-critical function of our application still works error-free even under a significantly noticeable base load.

So stay tuned for the following post here in my blog.

Note: You can find my german original here

Bibliography

1. https://gettaurus.org/install/Installation/ - Installing and Upgrading
2. https://gettaurus.org/docs/DataSources/ - Data Sources
3. https://gettaurus.org/docs/Gatling/ - Gatling Executor