No matter how experienced you are, it’s easy to get terms mixed, confused, and misunderstood. To help, we’ve prepared this brief primer on performance testing vs stress testing to define these common terms (and a few others). You may still hear some of these terms used interchangeably, but at least now you have ammunition to correct others when you’re feeling pedantic. <grin> Let’s start with load testing.
Load testing is testing your site, API or app under a particular load (thus the name). Specifically, it’s testing how your entire system operates under the simulated pressure of a particular number of users performing a particular number of tasks. Designed to simulate real-world use, many people equate load testing with testing of a particular number of simulated users.
In a sense, it’s a way to make sure that your expected level of users gets the best possible experience. Sometimes, we use “load testing” to encompass all of these performance-related tests, but it’s generally best to use the term “load testing” to refer specifically to testing the system load a set of virtual users places on the system.
Load testing, too, is what we recommend you integrate into your continuous delivery practice (CD) so that you’re watching for any issues that may crop up in any part of your project in regular builds - code, databases, architecture - any of the many areas today’s modern developer/devops pro has to watch for.
In contrast, the term “performance testing” can encompass more than the simulated user experience. Performance may watch for not only overall system performance under user load, but may also watch for things like data throughput, database I/O speeds, internal communication speeds, memory usage, and so forth.
The instrumentation for performance testing is related to load testing, but is not necessarily the same. The server agents that handle the load testing can communicate with other products (like NewRelic) to correlate performance issues as regular load tests run.
Stress Testing usually refers to a less frequent test that, to simplify, throws as much stress - usually simulated users and user actions - at an app/site/API as it can take. It’s not something you run frequently as you should a load test. Instead, it’s a way to illuminate weaknesses and bottlenecks in your projects before you’re anticipating an influx of traffic. (Or just as a regular preventative measure in case you do experience a traffic peak.)
Similarly, a “spike test” suggests that you’ve created a load testing scenario that simulates a brief traffic spike so that you can watch how your project responds and recovers. You may have also heard of a “soak test,” which is basically a long-running, slowly-ramping load test.
Choosing the Best Test
The best test, of course, is a consistently conducted load test as part of your CI / CD practice. As we advocate often, when you conduct load tests consistently, they illuminate potential performance issues before they can become a problem. Of course, you should also correlate those tests with your regular performance testing to watch for bottlenecks. Stress testing is an excellent, less-frequent way to battle-harden your projects against unanticipated usage.
In this short space, we’ve oversimplified some, but this should give you a good working definition of each type of test - and when you should use them.