Every SmartMeter.io test report now contains a comparison of results from previous test runs to give you a clear overview of performance trends. Let’s take a look.
The new Trend Analysis tab in test report contains a table and charts with a comparison of relevant metrics from previous test runs.
Trend analysis table
The table shows how the results changed from previous test run to give you an instant overview.
The maximum number of reached virtual users
Average reponse time
Hits per second
For each transaction in test scenario there is a comparison chart for following metrics:
Average Response Times
Median Response Times
How it works
The Trend Analysis feature works out of the box. Please note that the test report must be created in SmartMeter.io 1.4.0 or newer! Trend Analysis is available even in the free SmartMeter.io Light with limitation to 3 test runs.
SmartMeter.io automatically detects previous test runs in report folder and uses them for comparison. Test runs are grouped by test script name (the .jmx file). In case of major changes in the test script we recommend to version its name to start over.
If you want to exclude a test run from comparison, either remove the whole report folder or delete the data/test-summary.json file from it. Then generate the last report again.
Do you find this feature helpful? Do you miss some data in Trend Analysis? Let us know in the comments.