Understanding your load testing results can be a challenge. Automated results analysis can really cut down on the time you spend poring over your test data to figure out where the bottlenecks are. In our user survey, we found that automated results analysis is the second most wanted capability for a load testing solution.
Performance Alerts are the newest automated results analysis feature in Load Impact Insights. With Performance Alerts, you can quickly see where you have a performance issue either in your system under test or in your test scripts.
Load Impact automatically analyzes your performance test results when you stream your data to Load Impact Insights (k6 run -o cloud script.js) or execute your test in the Load Impact SaaS application. (Learn more about Load Impact Cloud Execution).
There are two categories of alerts: Smart Result Performance Alerts and Test Health Performance Alerts.
Smart Result Performance Alerts
Smart Result Performance Alerts analyze your performance test results and alert you to performance issues in the system under test. There are currently 2 alerts in this category:
Your target system (website, web app, API) has hit a throughput limit, meaning the request rate has flat-lined. Your system is overloaded.
Figure 1: Throughput Limit Performance Alert Example
High HTTP Failure Rate
The HTTP error rate for some period during the load test is at least 10% higher than at the beginning of the test. It usually means the target system is close to its performance limit.
Figure 2: High HTTP Failure Rate Performance Alert Example
Test Health Performance Alerts
These alerts highlight test or script related issues. These issues can either skew your performance test results or make analysis harder. You can often quickly resolve these issues through changes in your test script. There are currently 4 alerts in this category:
Third Party Content
Insights detects more than 3 different domains in a test. This is typically caused by your test script containing requests to 3rd party resources such as CDNs, social media scripts, analytic tools, etc. That should usually be avoided.
Figure 3: Third Party Content Performance Alert Example
Too Many URLs
Insights detects more than 500 unique URLs in your test results. Commonly caused by a URL that contains a query parameter that is unique per iteration, e.g. tokens, session IDs, etc. Use the URL grouping feature in k6.
High Load Generator CPU / Memory Usage (2 separate alerts)
These alerts tell you if your test configuration is causing over utilization of the load generator resources (CPU and Memory), which can skew your test results.
Learn More About Performance Alerts in our Knowledge Base
Don't miss our webinar on Automated Load Testing Results Analysis. See a live demo of Load Impact Insights Performance Alerts.
Date and time:
Thursday, Nov 28, 2018 @ 1pm ET
This webinar is presented by Robin Gustafsson, CTO and Mark Meier, Team lead Customer Success, at Load Impact.