Performance Trend Analysis: Compare Load Test Runs at a Glance

Every SmartMeter.io test report now contains a comparison of results from previous test runs to give you a clear overview of performance trends. Let’s take a look.

trend-analysis-load-test-report-smartmeter-io

The new Trend Analysis tab in test report contains a table and charts with a comparison of relevant metrics from previous test runs.

See an example report here

Trend analysis table

The table shows how the results changed from previous test run to give you an instant overview.

trend-analysis-load-test-report-smartmeter-io-tableThe table contains following metrics:

  • The maximum number of reached virtual users

  • Test duration

  • Average reponse time

  • Total errors

  • Total hits

  • Hits per second

See an example report here

Comparison charts

trend-analysis-load-test-report-smartmeter-io-chart

For each transaction in test scenario there is a comparison chart for following metrics:

  • Average Response Times

  • Median Response Times

  • Error Rate

| See an example report here

How it works

The Trend Analysis feature works out of the box. Please note that the test report must be created in SmartMeter.io 1.4.0 or newer! Trend Analysis is available even in the free SmartMeter.io Light with limitation to 3 test runs.

SmartMeter.io automatically detects previous test runs in report folder and uses them for comparison. Test runs are grouped by test script name (the .jmx file). In case of major changes in the test script we recommend to version its name to start over.

If you want to exclude a test run from comparison, either remove the whole report folder or delete the data/test-summary.json file from it. Then generate the last report again.

Do you find this feature helpful? Do you miss some data in Trend Analysis? Let us know in the comments.

Download current version of SmartMeter.io