Automated Load Testing Results Analysis

Posted by John Emmitt on Nov 14, 2018

Understanding your load testing results can be a challenge. Automated results analysis can really cut down on the time you spend poring over your test data to figure out where the bottlenecks are. In our user survey, we found that automated results analysis is the second most wanted capability for a load testing solution. 

Performance Alerts are the newest automated results analysis feature in Load Impact Insights. With Performance Alerts, you can quickly see where you have a performance issue either in your system under test or in your test scripts.

Load Impact automatically analyzes your performance test results when you stream your data to Load Impact Insights (k6 run -o cloud script.js)  or execute your test in the Load Impact SaaS application. (Learn more about Load Impact Cloud Execution).

There are two categories of alerts: Smart Result Performance Alerts and Test Health Performance Alerts.

Smart Result Performance Alerts

Smart Result Performance Alerts analyze your performance test results and alert you to performance issues in the system under test. There are currently 2 alerts in this category:

Throughput Limit

Your target system (website, web app, API) has hit a throughput limit, meaning the request rate has flat-lined. Your system is overloaded.

throughput-limit-perf-alert

Figure 1: Throughput Limit Performance Alert Example

 

High HTTP Failure Rate

The HTTP error rate for some period during the load test is at least 10% higher than at the beginning of the test. It usually means the target system is close to its performance limit.

too-many-http-failures

 Figure 2: High HTTP Failure Rate Performance Alert Example

 

Test Health Performance Alerts

These alerts highlight test or script related issues. These issues can either skew your performance test results or make analysis harder. You can often quickly resolve these issues through changes in your test script. There are currently 4 alerts in this category:

Third Party Content

Insights detects more than 3 different domains in a test. This is typically caused by your test script containing requests to 3rd party resources such as CDNs, social media scripts, analytic tools, etc. That should usually be avoided.

third-party-content-perf-alert

Figure 3: Third Party Content Performance Alert Example

Too Many URLs

Insights detects more than 500 unique URLs in your test results. Commonly caused by a URL that contains a query parameter that is unique per iteration, e.g. tokens, session IDs, etc. Use the URL grouping feature in k6.

High Load Generator CPU / Memory Usage (2 separate alerts)

These alerts tell you if your test configuration is causing over utilization of the load generator resources (CPU and Memory), which can skew your test results.

 Learn More About Performance Alerts in our Knowledge Base

 

Don't miss our webinar on Automated Load Testing Results Analysis. See a live demo of Load Impact Insights Performance Alerts.

Date and time:

Thursday, Nov 28, 2018 @ 1pm ET

This webinar is presented by Robin Gustafsson, CTO and Mark Meier, Team lead Customer Success, at Load Impact. 

Webinar registration

 

 

 

Topics: performance issues, performance test, Automated results analysis, load tests

Recent Posts

Popular posts

Posts by Topic

see all

Subscribe to Email Updates