Load Testing for the Mobile Age

Photo by Jan Vašek. Courtesy of StockSnap.io

As we all know, load testing matters.

It’s one thing for an application to work for 10 customers, and it’s quite a different thing for it to perform well with 1,000.

What many companies aren’t aware of is that testing is important for more than just your on-premises applications.

The growth of cloud applications means you have to pay attention to performance even when you’re not hosting inside your firewall.

An even bigger issue that many people still haven’t recognized is the growth of application access via mobile devices: Smartphones and tablets.

Wireless Tests Differently than Wired

The wired world has been well understood for years. Load testing can incorporate estimates of latency for global testing because the knowledge of the internet improves the predictability of tests.

However, wireless access adds a new set of issues that many IT organizations haven’t considered. For instance, response through a busy cell tower in a downtown area could be much slower than access from a suburban location.

Multiple generations of mobile devices are in use, each capable of different access speeds and even weather can have an impact on performance.

Load Impact helps you by focusing on the critical issues in mobile performance:

  • Network Delay: How fast a single data packet can travel from A to B on the internet. Today, it is more or less dependent on the physical distance between the two machines communicating and the speed of light in a fiber optic cable
  • Network Bandwidth: How many bytes can be transferred each second between A and B on the Internet. This is dependent on the lowest available bandwidth on any physical link on the route between A and B. How much bandwidth is available on a physical link depends, in turn, on the transmission equipment used and how much other traffic is currently using the same link
  • Packet Loss: How large a percentage of transmitted packets get lost in transit. Congested physical links or routers may discard packets. Packet loss today is in most cases very low, but for radio links (mobile, WiFi) or for heavily congested links it can be substantial
  • Jitter: The variation in network delay. Some delay-sensitive applications have ways to work around situations where they encounter high network delay, but if jitter is high and it causes one packet to take 100 ms (for example) to travel from A to B while the next packet takes 10 ms, that makes life a lot harder for the application (in that example, packet #2 would arrive before packet #1)

Why does that matter?

Take, for instance, many of the new active graphics used on web pages. A change in latency – from 2ms (broadband) to 400ms (3G network) – can cause a page load time to go from 1 second to 30 seconds.

You need to be able to test how pages react under different scenarios to ensure that you respond to mobile users in a way that enhances their experience.

It’s not just Applications: Websites and Search

The user experience isn’t only an issue for your current customers. Internet searches matter and the better you can provide prospects with information the better you can fill your lead funnel. That means optimizing your web pages, not just for once they find you, but for search engines.

As mentioned in a Load Impact March blog post, Google is now using the speed of response for a web site in its search rankings. Load testing is critically important for companies who want a first-page ranking to drive a lot of visitor traffic. Along with a general position on web site speed, Google is paying attention to how well sites are optimized for smartphones and tablets.

Google’s Jerry Dischler, Vice President, Product Management, AdWords, pointed out that there are more Google searches on mobile devices than on standard computers in 10 countries, including the US. That’s one reason web searches from mobile devices now are ranked on adaptability for mobile, including performance.

People are using the web more and more to find information and they’re doing so while one the go. If you aren’t testing your web site’s performance in the wireless world, you’re likely missing out on business.

Integrated Load Testing

Notice the focus on mobile? Yes, it is very important, but that doesn’t mean hardwired connectivity is going away anytime soon. Every avenue of access can impact another.

For instance, if a smartphone has a poor connection to a database and has key information locked from access, how many other customers will be delayed while that single mobile user accomplishes a task?

You need to ensure that mobile and wired load testing is done together, to provide true load testing for the real world.

Summary

The balance of access to searches and website access is clearly tilting toward mobile customers.

Meanwhile, more and more applications are running on the same devices. Mobile testing must be incorporated into the development cycle in order to ensure that users, regardless of how they reach you, have good user experiences and so your own teams can do their jobs wherever they may be.

About the Author

David A. Teich is principal consultant at Teich Communications, a technology consulting and marketing services company. Teich has worked in the business of technology for more than three decades, and he has experience in marketing, sales, professional services, development and operations.

Through Teich Communications, Teich provides strategic marketing, product positioning and writing services designed to help small and midsize technology vendors better understand and communicate the fit between their products and market needs.

In addition to his industry experience, he has an undergraduate degree in business computing from Texas A&M University, a master's degree in computer science from Stanford University and an MBA in marketing from Pepperdine University.

Read More

[NEW RELEASE] Mobile Performance Testing - Including Network and Client Emulation

Today, we introduced the first true cloud-based load testing Software as a Service for mobile apps, APIs and websites that can simulate traffic generated from a variety of mobile operating systems, popular browsers, and mobile networks – including 3G, GSM and LTE.

Read More

Mobile Network Emulation - The Key to Realistic Mobile Performance Testing

When was the last time you looked at your website's browser statistics? If you have, you've likely noticed a trend that's pretty hard to ignore - your users are browsing from a mobile device more than ever before. What was once a small sub-segment of your audience is now growing and representing the majority of your traffic. This may not be so surprising since today mobile usage makes up about 15 percent of all Internet traffic. Basically, if you don't already have a mobile development strategy, you may already be loosing sales/users due to poor mobile performance.

Responsive design takes care of your website's layout and interface, but performance testing for mobile devices makes sure your app can handle hundreds (even thousands) of concurrent users. A small delay in load-time might seem like a minor issue, but slow mobile apps kill sales and user retention. Users expect your apps to perform at the same speed as a desktop app. It seems like a ridiculous expectation, but here are some statistics:

  • If your mobile app fails, 48% of users are less likely to ever use the app again. 34% of users will just switch to a competitor's app, and 31% of users will tell friends about their poor experience, which eliminates those friends as potential customers. [1]
  • Mobile app development is expected to outpace PC projects by 400% in the next several years. [2]
  • By 2017, over 20,000 petabytes (that's over 20 million gigabytes!) will be sent using mobile devices. Streaming is the expected primary driver for growth.[3]
  • 60% of mobile failures are due to performance issues and not functional errors. [4]
  • 70% of the performance of a mobile app is dependent on the network. [5]
  • A change in latency from 2ms (broadband) to 400ms (3G network) can cause a page load to go from 1 second to 30 seconds. [6]

These statistics indicate that jumping into the mobile market is not an option but a necessity for any business that plans to thrive in the digital age. You need more than just a fancy site, though. You need a fast fancy site. And the surefire way to guarantee your mobile site/app can scale and deliver a great performance regardless of the level of stress on the system is to load test early and continuously throughout the development process.

Most developers use some kind of performance testing tools during the development process. However, mobile users are different than broadband users and therefore require a different set of testing tools to make sure they are represented realistically in the test environment. Mobile connections are less reliable; each geographic area has different speeds; latency is higher for mobile clients; and older phones won't load newer website code. Therefore, you need real-world mobile network emulation and traffic simulation.

Prior to the availability of good cloud performance testing tools, most people thought the solution to performance problems was "more bandwidth" or "more server hardware". But those days are long over. If you are to stay competitive today, you need to know how to optimize your mobile code. Good performance testing and traffic simulations take more than just bandwidth into account. Network delays, packet loss, jitter, device hardware and browser behavior are also factors that affect your mobile website’s or app’s performance. To properly test your app or site, you need to simulate all of these various situations - simultaneously and from different geographic locations (i.e. not only is traffic more mobile, its also more global).

You not only want to simulate thousands of calls to your system, you also want to simulate realistic traffic behavior. And, in reality, the same browser, device and location aren't used when accessing your site or app. That's why you need to simulate traffic from all over the globe with several different browsers and devices to identify real performance issues. For instance, it's not unlikely to have a situation where an iPhone 5 on the 4G network will run your software fine, but drop down to 3G and the software fails. Only realistic network emulation covers this type of testing environment.

Finally, simulating real user scenarios is probably the most important testing requirement. Your platform's user experience affects how many people will continue using your service and how many will pass on their positive experience to others. Real network emulation performs the same clicks and page views as real users. It will help find any hidden bugs that your testing team didn't find earlier and will help you guarantee that the user experience delivered to the person sitting on a bus using a 3G network is the same as the individual accessing your service seated at their desktop connected through DSL.

Several years ago, mobile traffic was negligible, but it's now too prominent to ignore. Simple put, don't deploy without testing your mobile code!

Check out Load Impact's new mobile testing functionality. We can simulate traffic generated from a variety of mobile operating systems, popular browsers, and mobile networks - including 3G, GSM and LTE. Test your mobile code now!

Read More

Posts by Topic

see all
Try a Free Test