How to Automate Load Testing

Posted by Load Impact on Oct 3, 2017

Most developers are extremely familiar with automated testing as part of their CI/CD pipelines. Continuous improvement and continuous delivery integrate unit testing as a matter of course.

But there’s another type of testing you *should* be doing, but most developers aren’t. Since we’re load testing experts, we’re sure you’ve already guessed our recommendation: add load testing to your automation pipelines.

While unit testing is commonplace, load testing isn’t. You can not only differentiate your code performance by integrating it, but also catch tiny performance issues before they become big ones simply because you’re load testing consistently.

Here, we’ll share some general strategies for integrating load testing into your automation pipeline.

scaffolding-1617969_640.jpg

First, we’re assuming you have an existing CI/CD tool like Jenkins or Circle CI. Any of those tools can trigger an external process. In this case, you’d trigger either LoadImpact or k6 by LoadImpact. (For specific steps for any tool, check out our integrations pages for LoadImpact here, and k6 by LoadImpact here.) So our first tip is to make sure you are familiar with how to set up the automation tool you prefer - and what notification process you want to use if it’s external to that tool (we use Slack).

Second, determine the frequency of your load tests. A unit test or functional test can run very quickly - usually in less than a few minutes. Those are easy to run with every build, and should be run with every build. They’re fast enough that you can run them with every commit.

But a load test is different. A good, thorough load test can take time to run: 10 minutes or much longer. That’s too long to wait when you’re committing code more than once a day (and you likely are).

Instead, you should run your load tests once a day, preferably with overnight builds. That’s exactly what we do, and the load test scripts are triggered with the build script. When we arrive to work in the morning, we know not only how our unit and functional tests ran, but also exactly how the performance tests behaved. Armed with this information, we can troubleshoot any performance issues.

Third, make sure you remember to set performance thresholds for your automated testing. Here’s where the pass/fail message capability comes into play. For example, using k6 by Load Impact, you can receive a pass/fail message after the automated load test back to your CI/CD tool - and with a ‘fail’ message, the build itself can fail.

For example, you may want to set a performance goal that the performance cannot degrade more than a certain amount (say, 5%) without failing the build. Of course, you may want to be more strict and not allow any performance degradation in order for the build to proceed.

With those strategies in mind, you should be ready to plan your automated load testing.

Topics: DevOps, continuous delivery, test automation, continuous integration, Load Testing, jenkins

Popular posts

Posts by Topic

see all