Whether you’re new to DevOps or an experienced hand, sometimes it’s a challenge to explain the difference between performance testing and performance tuning. It happens to us, too. Here’s how we describe understanding performance testing and performance tuning.
For the most accurate load test, best practices (and common sense) dictate that your testing environment be as close to real production and user behavior as possible. We get it, though! Best practices don’t necessarily reflect the challenge and rush of a real development pipeline. For example, if you’re like us, you likely have a much larger amount of data in your production environment than in your testing and staging environment. A detailed, extremely realistic testing database is nice, but perhaps not necessary. Let us explain.
We have created a simple step by step guide on how to integrate k6 in your TeamCity build setup.
For all you CircleCI users out there, we have created a simple step by step page on how to integrate k6 in your build setup.
For all you Jenkins users out there, here's a guide on how to integrate k6 in your Jenkins Pipline setup.
As we’re big believers in dogfooding we’ve used our open source load testing tool, k6, to set up load testing automation with CircleCI. This is how we did it, and some tips and tricks to make your load testing automation easier. Let’s go!
Photo Courtesy of StockSnap.io
Faced with the growing appetite for innovation, business agility and cost savings, progressive organizations rally around DevOps-driven values of collaboration, quality and automation.
These capabilities are key to accelerating growth and enabling valuable competitive differentiation with high quality software delivered and deployed at the rapid speed of business demand within budget.
These unprecedented efficiencies result from the infusion of test automation tools and practices as a holistic approach to embracing the true spirit of DevOps in the form of continuous testing, integration, delivery, release and deployment.
Why Automate Testing?
Manual tests designed to exercise and test application functionality compare actual results to ideal or desired performance and report the outcomes. These tests are typically conducted on a frequent basis every time the application changes.
In some cases, QA has to repeat these tests for various combinations of system configurations, operating systems and component integrations.
Manual intervention and limited human capacity lead to error-prone testing procedures that disrupt agility in deploying software applications.
Software defects and costly delays are inherent in traditional manual testing practices when the code is available for testing only after developers throw it over the wall.
QA tests the code for defects and returns it back for developers to try and apply quick fixes, build another version and send it back for testing. This process repeats and forms a death spiral in action that forces Ops to tolerate some software defects and release the application into production.
Each iteration introduces additional technical debt and impacts software quality, functionality, reliability and increases the workload for developers, Ops and QA alike.
The ability to automate repeatable testing practices is critical for QA and Ops to catch up with the fast pace of Agile development sprints and bring agility to the DevOps testing cycle.
Automation Drives the DevOps Testing Cycle
The DevOps methodology aims to bring order to the otherwise complex and chaotic world of software development, testing and production.
The concept eradicates turf wars among siloed departments and independent teams that must work collectively to enable productivity throughout the application lifecycle.
Eliminating siloization among working groups and establishing meaningful connections to facilitate productive and collaborative relationships fosters innovation as much as organizational culture.
The true essence of the DevOps movement lies in providing the right tools and mechanisms that effectively enable this innovation and cultural change.
The potential of continuous testing is gaining steam in revolutionizing DevOps efforts. The practice requires DevOps teams to shift testing to the left, beginning early during the development lifecycle and spanning quality assurance across the delivery pipeline.
The enhanced test coverage and continuous feedback to all stakeholders enables informed decisions, removes waste and strikes an optimal balance between speed and quality.
The DevOps end-goal of automated and continuous deployment is practical only as long as each iteration yields consistent improvements.
The notion of quality is weaved into the fabric of DevOps, which relieves QA's responsibility of constantly identifying defects and starts a process of avoiding many of them in the first place.
Continuous testing in the DevOps world has to start with developers leveraging automated performance testing tools to ensure the entire code is error-free by the time it is delivered to QA.
This is the first major step toward continuous delivery as the application is built, packaged and deployed into a production-like environment for testing. Provisioning new instances for this QA environment is typically expensive, complex to maintain and is limited in availability.
These issues in turn necessitate further automation, in the form of infrastructure automation and service visualization for rapid provisioning and horizontal scaling of powerful and dedicated resources to execute all tests in parallel.
Delivering every iteration to a staging environment using automated systems enables push-button deployment into production as soon as the organization is ready.
Automating with Load Impact
To embrace the radical new DevOps-driven paradigm shift of supporting automation and continuity across all testing practices, organizations must empower developers, Ops and the QA police with next-generation test automation solutions.
The business value of test automation comes down to capabilities that effectively accelerate end-to-end application delivery at low cost and against aggressive timelines.
By integrating with the Load Impact API, DevOps-driven organizations can identify defects faster and accurately, perform quick fixes and pursue unprecedented performance improvement opportunities in deliver high quality applications at accelerated speeds while maximizing profitability.
— Ali Raza, a contributing writer for Load Impact's blog, is a technology consultant and research analyst with leading Fortune 500 companies, promising new startup firms and major international publications. As a racing driver and a stunt master, only cars eclipse his love for technology.
Starting today, we're accepting applications for early access to our new Load Impact Continuous Delivery service, which for the first time allows developers of websites, apps and APIs to make performance testing an integrated part of their continuous delivery process.
