You may have heard that the holiday season is approaching.
If you’re working for a B2C company, there’s a good chance this is a busy time of year for you, and you’re expecting more web traffic than usual.
Creating realistic load tests that mirror your the checkout process is absolutely imperative when preparing your websites, apps, APIs and infrastructure for load tests.
So, here are some processes to think about that will help you get the most actionable data from your tests. And that leads to better optimizations, which leads to better performance, which leads to happier customers and more revenue.
Intro to Data Stores
In order to include randomized data to make your load tests realistic, you’ll need to utilize Data Stores — which is a term for parameterized data created from a CSV file. Here’s more information from the Load Impact Knowledge Base, and we also have a quick video if you’re more of an audio/visual learner.
Now, let’s get on with the juicy bits of user scenario script building for your e-commerce checkout process.
You’re selling something, and there’s a good chance you use a payment provider to collect information (credit card info, etc.)
These solutions typically provide some kind of QA/test environment with a number of possible payment methods, but it’s important to know that many payment providers don’t want you to include their service in your load tests.
Simply put, payment providers don’t want you running a load test with hundreds of thousands of Virtual Users and crashing their product. (Seems like a reasonable request from them)
If you think it’s absolutely imperative to test the actual payment-transaction process in your load test, you should probably notify your payment provider first.
In some cases, creating simulated requests against your payment provider’s system could violate your terms of service, as these companies are trying to protect themselves from performance issues.
So, take a look at your TOS or just send an email to your payment provider’s support team to make sure you’re in the clear.
Email or other Communication
People usually expect to receive some kind of email confirmation after making a purchase, and many of our customers have asked about including that in their load tests.
Our advice is to set up a catch-all mailbox to receive all those test emails.
A load test can generate thousands of emails (depending on the number of Virtual Users you’re testing), and sending all of those in a short time frame without configuring your email settings could cause you to be blacklisted or flagged as spam by an email provider.
Also, if your purchase confirmation is sent via SMS, then you’ll probably want to just disable that when running your load tests. Depending on how you pay for that, sending thousands of test messages could be a waste of money.
This is a big one for our e-commerce friends working with a supply chain.
If you’re including real items in your test, Virtual Users running in your test scenario can cycle through the scenario several times, which means they’ll just keep “ordering” an item until the test finishes.
Depending on the level of automation you’re using in your logistics, this could lead to your system automatically re-ordering items you don’t need.
So, make sure you’re either testing something with plenty of inventory, or go ahead and configure some items specifically for your load tests that won’t “run out” of inventory or trigger an auto-reorder.
Most of our e-commerce users will create a few fake SKUs to only be included in load tests to accomplish this.
Backend Integrations (Logistics, etc.)
In the same vein as inventory, you’re probably using other integrations — invoicing, logistics providers, printing, CRM, financials, etc. — that could be affected by hundreds or thousands of Virtual Users accessing your website and “ordering” things.
Keep all of those in mind when creating your user scenarios and tests. It could save you from a massive (and unnecessary) headache down the line.
Load tests consume data very quickly. You definitely want to make sure your Data Stores have plenty of information for the Virtual Users in your load test.
Let’s outline a quick breakdown to show what we mean. Let’s say you’re running a test with this configuration:
- Ramping up to 5,000 concurrent Virtual Users
- Test duration of four minutes
- Testing process of user logging in with a username and password, searching the product listing, adding an item to a cart and checking out
- Various user scenarios take around two minutes each to complete
So, if you want every scenario to be executed with a unique username and password, you’re going to need to include about 10,000 lines in the CSV file you convert to a Data Store.
Based on the scenario above, each user scenario will be completed about twice within the test duration, so that’s why you’ll need about 10,000 unique rows in your Data Store.
Tokens: CSRF Tokens, VIEWSTATE, etc.
This can be an unforeseen wrinkle for the average engineer who hasn’t been working specifically in QA and performance testing throughout their career.
For many of our users who utilize the Load Impact User Scenario Recorder Chrome extension to easily create testing scripts, the hiccup starts when our scenario recorder captures their specific CSRF token.
In order to properly test unique CSRF tokens for each of your Virtual Users in your test, you’ll need to extract the values, save it to a variable and concatenate it into the future relevant requests.
There’s a bit of scripting involved in that, but here’s an easy reference guide (with a theoretical example!) to help you load test a website with CSRF tokens or VIEWSTATE.
And there you have it. Hopefully some of those tips can help you write better load tests for your e-commerce sites this holiday season.
If you have any questions, check out our Knowledge Base or hit us up any time. Our Customer Success team is happy to answer questions from any user.