Load Testing Execution Phase 2: Benchmarking and Complex Cases

Posted by Grant Engle on Dec 8, 2016

In the previous article in this series, we talked about getting prepared for your performance testing by:

  • Creating user scenarios
  • Configuring and running smoke tests
  • Creating load tests

At this point, you’re onto the actual testing. You’re running load tests and finding actionable data in the results.

We’ve broken this phase down into six parts, but it’s important to remember that each part may require multiple iterations. But hey, multiple iterations of each step just means you’re continuously finding new features to optimize or new problems to fix, and that will only improve the user experience in the long run.

crowd grand central black and white.jpg

 

Step 1: Load tests

Based on the goals you set during the creation of your user scenarios and the configuration of your tests, you’re now armed with expectations when you start running these load tests.

When reviewing the results of your load tests, you’ll identify bottlenecks, failures and edge conditions, as well as establish a baseline for evaluating the impact of future changes to your system.

So, these tests will show you how close you are to achieving your performance goals, and they can also inform the goals you set in the near future.

Step 2: Max Capacity Tests

This series of tests will help you determine the maximum number of concurrent users your application and infrastructure can accommodate while meeting your performance expectations.

When configuring these tests, we encourage you to push the boundaries of your system so you really get a sense of how it’ll behave under intense pressure.

Step 3: Stress Tests

Once your system has been tuned to its optimum state, let’s see if we’re able to break it again!

Stress tests find your system’s breaking point and reveals what happens when it breaks. We know it’s against human nature to intentionally break something you built (normally), but the data you’ll get from this test is too valuable to your long term planning for you to be handling your app and infrastructure with kids’ gloves.

So, go ahead: Break your website and learn when that can happen and what it means!

Step 4: Configure New Tests

As you probably guessed, there’s a lot to learn from those first three rounds of tests. You’re likely going to have a fairly long list of tweaks and optimizations you’ll need to make.

After you make those tweaks and optimizations, it’s time to run some of those tests again to verify that your changes worked. You’ll also come across new information along the way that inspires you to run more tests with different user scenarios and configurations. Run those, too!

Repeat steps 1-4 until you’ve reached your performance goals and have a clearer vision of how you want your performance to evolve moving forward.

whale big water robust.jpg

Step 5: Soak Tests

Depending on your service and the complexity of its backend — systems with complex workflows, document management, logistics controls, etc. — you might want to run a series of soak tests.

These tests uncover performance bottlenecks stemming from a system being under pressure for an extended period of time.

Issues such as memory leaks, resource leaks or corruption and degradation that occur over time can be identified and optimized based on data accumulated from soak tests.

Step 6: Robustness Tests

This step doesn’t necessarily apply to everyone — mostly larger entities like banks, data management companies, etc. — but it’s an important final step for those entities.

Robustness tests are also commonly referred to as “functional tests under load” by some performance professionals.

Basically, this is testing with the specific goal of verifying that a system will continue to to function as expected when put under abnormal pressure.

These tests are especially useful for evaluating failover and disaster recovery procedures.

Entering Phase 3

Now you’ve run many tests, learned many new things, made plenty of optimizations and you’re feeling pretty great about your system’s performance moving forward.

Why not keep the good vibes going by implementing performance testing into your Continuous Delivery process?

That’s what we’ll tackle in the final blog post of this mini-series.

Stay tuned!

Topics: Performance testing, stress testing, stress test, Load Testing, benchmarking, soak testing, soak test

Posts by Topic

see all