Performance Testing for the 2016 Olympics

Posted by Grant Engle on Aug 3, 2016

The 2016 Summer Olympics will be the most-watched television event in the world for two weeks.

The engineering team at USA Gymnastics knew it was time to ramp up performance testing, and Load Impact is proud to partner with them to make sure their website and infrastructure will be ready to handle the prolonged uptick in traffic.

USA Gymnastics improved average load times on its most important pages by nearly 100 percent across the board, and they got there by analyzing data from their Load Impact tests.


Most organizations have compelling events throughout the year that require them to pay extra attention to performance testing. It could be a big sale, an ad campaign, a popular marketing video or even a celebrity endorsement.

USA_Gym_logo.png

But what if you needed to prepare for the biggest sporting event in the world — and you expected a major, consistent boost in traffic for about two weeks?

That’s exactly what the engineers at USA Gymnastics knew they were facing leading up to The 2016 Summer Olympics in Rio.

Average load times for some of the most important pages on the website were taking as long as a full minute to load under the stress of 1,000 concurrent users for extended periods of time, and that was simply unacceptable for the fans and members of the organization.

Mike Bowman, IT manager at USA Gymnastics, knew the site could be optimized for peak-performance and maintain a modern look. He just needed the right testing software.

“We knew our website couldn’t handle a large hit last year,” Bowman said. “We tried a few things, but we needed detailed results from testing that could help us optimize our website, server and database bottlenecks.”

The Testing Project

USA Gymnastics started their testing project by looking at website traffic from the 2012 Summer Olympics to help them gauge what they could expect this year.

Bowman and his team determined that they could expect a little less than 1,000 concurrent users hitting their website at several different times throughout the two-week Olympics, so he based their testing on that.

It’s typically a good idea to prepare for slightly higher traffic than you expect, so Bowman decided that the majority of tests would include 1,000 concurrent VUs to be on the safe side.

Working alongside the Load Impact Professional Services team, Bowman established benchmarks and goals for website, server and database performance.

Then, the testing and optimization phase began.

“From Load Impact test results, we were able to see a lot of activity in the database server that we weren’t seeing before,” Bowman said. “That helped us find the bottlenecks and start making tweaks to correct them.”

While some engineers think about a compelling event generating a ton of visitors for a short period of time, the engineers at USA Gymnastics have a somewhat unique situation.

“The Olympic Qualifiers actually had a bigger one-day impact than any one day of the last Olympics,” Bowman said. “But we needed to prepare for a constant stream of traffic for the entirety of the Olympics — which is two weeks of consistent elevated load.”

USA Gymnastics ran long tests with moderate ramp-ups to simulate their expected scenario for the Olympics. Their strategy was to understand how the website, web server and database would react to a consistently high flow of visitors.

spike_test_vs_load_test.png

A load test (left) vs. a spike test (right)

The Results

When Bowman and the other USA Gymnastics engineers started testing with Load Impact, they found that their database server was overloaded with 1,000 concurrent virtual users (VUs).

That was obviously not good, but using the metrics from Load Impact tests, Bowman and his team were able to dive into what was causing problems.

After multiple test runs of a few user scenarios, USA Gymnastics was able to see improvements as they continued to optimize with our Performance Trending feature, and that eventually led them to see how they reached their performance goals.

USA_Gymnastics_average_load_time-1.png

As you can see from the chart above, average load times for the critical pages on the USA Gymnastics website decreased by nearly 100 percent!

Scott Allen is a web developer at USA Gymnastics who worked with Bowman on this project. He’d never worked with performance testing software before, but he said the solution to their problem was clear pretty quickly.

“After running the first version of the test, the entire database server bogged down,” Allen said. “The solution became apparent that we needed to spin up another server with a lot more CPU and RAM.”

The engineers also made a few other code changes to remove some unnecessary database connections on some of their public pages, which they discovered from their test results.

Bowman and Allen made the adjustments, and now they have empirical data to prove they’re ready for the expected bump in traffic — and then some.

Lessons Learned

USA Gymnastics did a lot of things right in this engagement. So, here’s what they did and how you can do it yourself:

  • Start testing early
    • USA Gymnastics started testing more than two months before the opening ceremony of the Olympics
    • They gave themselves plenty of time to test early and continuously
  • Clearly define your goals
    • Bowman and his team looked at their traffic data from a similar event — the 2012 Summer Olympics — and extrapolated from there to determine how many VUs to use in their tests
    • They focused on improving load times as they tested and tweaked — they didn’t only focus on the end goal of a specific load time
  • They knew what they didn’t know
    • USA Gymnastics is obviously a team of talented engineers, but performance testing is a niche discipline, so they made sure to engage our Professional Services team to help them get started
  • They took this opportunity to become great testers
    • Bowman and his team became excellent performance testers quickly. They’ve empowered themselves to continuously test, analyze and optimize application, server and database performance moving forward

Conclusion

The engineers at USA Gymnastics obviously understand the importance of the Olympics for the athletes, but they also know there are thousands of fans in the U.S. and around the world who are going to flock to their website as the games are played over those two weeks.

There’s a massive difference in user experience when you consider some pages taking nearly a minute to load vs. page taking less than a second to load.

Bowman and his team have a responsibility to those fans to provide a great website experience, and they achieved that by approaching performance testing the right way and optimizing their site, server and database for peak performance.

For more information on how Load Impact data can help you optimize your applications, websites, APIs and servers, sign up for a free account today.

Topics: website optimization, Performance testing, Load Testing, USA Gymnastics, 2016 Olympics

Popular posts

Posts by Topic

see all