You’re ready to run your load test, you’ve recorded a script, and you know what you want to test.
But how do you choose the size of your load test? If you’re just trying out Load Impact, you’ll probably just choose the default test size for a free account. While that’s nice for testing the service, you’ll want to be much more specific for a real-world test.
The most crucial variable in your load tests is the number of concurrent users: in other words, the number of simulated virtual users (VUs) against which you’ll test. That number - the concurrent users - represents the peak of virtual users (VUs) who will be hitting your site or app at the same time.
Here’s how to estimate the number of virtual, simulated users you should test. First, consult your app or site analytics - be those Google Analytics or your app metrics service.
In those analytics, look for your average traffic over each month last year. Take a look at the monthly averages, and pay particular attention to months with increased traffic. To be safe, you should test against both the average daily traffic for your peak month, and the peak daily traffic during that peak month. The first test helps you gauge performance for a reasonable traffic peak in a normal month, and the second helps you gauge performance under significant load.
So, we’ve identified two concurrent VU numbers to test against: (1) average daily traffic in last year’s peak month, and (2) peak daily traffic in last year’s peak month.
If, for example, we know that e-commerce traffic is up by 20% year over year, which is true, at minimum, your peak tests should be at least 120% of those two numbers.
We also suggest a third test: take your peak daily traffic number and multiply it by 10 to test your project’s performance under unexpected, monumental stress.
As you can see, none of those three numbers are exact: they’re all estimates. So is Load Testing: we’re estimating capacity, simulating stress, all to help us identify weaknesses in our builds.
When you run your three tests with these three VU estimates, watch your site and app performance. Where does it degrade? And if it does so, does it do so gracefully? Does your site load time stay under 3 seconds - which is the general rule of thumb for an acceptable load time before you lose business?
Given the performance of those three tests, you can determine where you can expect degraded performance. You can also plan for when you may need to bring in additional caching or other resources in peak months, or in an emergency peak traffic event. You can also learn where your project might actually fail - and how to mitigate it.
Try these three VU concurrent user numbers in your next set of load tests, and let us know what it helps you discover.
PS: For a detailed look at choosing the number of concurrent users using Google Analytics, see this article: Determining Concurrent Users in Your Load Tests