Load Testing Execution Phase 1: Validation and Test Configuration

Posted by Grant Engle on Dec 1, 2016

As an engineer — or anyone in the working world these days — you have roughly one million things on your mind at a time.

So, we realize that after you’ve made the decision to load test with Load Impact (great choice, btw), that you might be interested in our 25+ years of load testing experience when it comes to getting started.

We recommend a three-phase process that you and your team will be able to manage pretty easily, and we’re going to outline each of those phases over the next three weeks in a mini-series of blog posts.Each phase will include a few steps, so let’s get started with Phase 1, Step 1.

winding road.jpg

Step 1: Create User Scenarios

A user scenario defines what URLs or web pages simulated users will request during a load test.

(Note: “User scenarios” can also be referred to as “load scripts” by some engineers. So, just remember these terms are interchangeable, but Load Impact almost always uses “user scenario.”)

The goal is to get actionable data from your performance testing, so your user scenarios must mirror realistic traffic patterns if you want to know which optimizations will have the greatest impact on load times.

Simply put: User scenarios are the backbone of your performance testing.

Check out this blog article on the best practices for creating realistic user scenarios for some guidance.

Our best users typically include multiple user scenarios in every test. That’s because no two groups of users are going to behave the exact same way, so you want to be sure your application and infrastructure can handle many different people doing different things at the same time.

blue smoke.jpg

Step 2: Smoke Tests

These tests are run in order to uncover obvious flaws in test scripts, platforms, code, etc. before committing the resources of a full test.

Run these tests with anywhere between 10-50 concurrent virtual users, and you might be surprised (pleasantly or unpleasantly) by what you can uncover.

It’s also important to note that when you upload a user scenario in the Load Impact app, that you’ll be asked to “validate” the scenario. That means we’ll run a single Virtual User through the scenario to make sure it doesn’t have any fatal errors.

While you should absolutely validate every user scenario you use before trying to run a test, it’s important to not consider that validation a “smoke test.”

Remember, a smoke test provides data on your app and infrastructure, too, while the validation only tests the user scenario.

Step 3: Create Load Tests

You have your user scenarios. You know they work. You’ve run your smoke tests and made any small optimizations that could have otherwise been show-stoppers.

Now, you’re ready to actually configure your load tests. Here’s what you need to know to do that.

  • How many concurrent virtual users do you want to test?
    • Rely on your average traffic data if you want to get your performance baseline
    • Crank it up 10x (or more) if you want to test the max capacity of your app and infrastructure
  • Which (and how many) user scenarios are you going to use in each test?
    • All of your users aren’t going to behave the same way, so multiple scenarios is very important — as we noted above
  • How will you allocate load between user scenarios?
    • You know all your users won’t be on the same journey, and this is where you configure
    • It’s usually a good idea to run multiple tests with differing percentages so you get a full picture of how your app could behave differently based on what people are doing
  • Where will load be generated from?
    • We have eight load zones from around the world to choose. Think about the regions where your traffic originates to answer this question
  • What browsers and networks should the test emulate?
    • Maybe the majority of your users are using Chrome over an unlimited network — or maybe they’re using mobile iOS over a 4G network. This is another example of where you should leverage your analytics and historic data to help you create a super-realistic test

Entering Phase 2

All right. The user scenarios and tests have been created. You’re feeling prepared, and you’re ready to take on the world of performance testing.

Stay tuned for our next blog post, which will focus on benchmarking and understanding some of the more complex cases that can arise when load testing.

Topics: Performance testing, Load Testing, smoke tests, scripting user scenarios, user scenarios

Recent Posts

Popular posts

Posts by Topic

see all

Subscribe to Email Updates