The Building Blocks for Successful Test Automation

As the digital marketplace continues its rapid transformation, companies are saddled with the unenviable task of keeping pace. Through no fault of their own, the demand for higher quality products and new features delivered faster than ever has created enormous pressure on engineering teams as they seek to maintain pace in a competitive marketplace.

As a result, automated testing has become a must-have as opposed to just a need. With an automated suite at your disposal, insurmountable tasks such as doubling the breadth of your code coverage, device coverage, and testing surface become tenable, cost-effective realities. The trick is implementing automation solutions that will hold up over time as you scale such efforts.

As we recently pointed out, there are several areas in which companies consistently struggle or fail when it comes to automation. Taking operational efficiency, cost management, risk reduction, and staffing limitations head-on as part of your automation rollout are the key to keeping your development pipeline running smoothly.

The Right Framework

It’s relatively easy to start an automation practice, because your coverage, scale, and risk is minimal. While getting started may not sound any initial alarms, teams can expect to hit trouble when amplifying their automation practice if they do not select the proper tools for their framework.

I’ve seen too many companies fail to identify a solution that comfortably fits their current needs while providing room to scale. For example, we did some consulting with a US-based eCommerce company that tried to build a large-scale, in-house mobile device lab as part of an initial automation effort for mobile app testing. This is an extremely complicated, expensive endeavor that could’ve been explored after a successful implementation with a mobile device cloud provider.

An Incremental Approach

Once you have your framework in place, it’s natural to move full speed ahead and transition a large portion of your manual functional testing into automated testing. Though it’s a noble thought, the transition to automation is rarely as smooth as you plan it to be. Tests are often inserted too quickly into the delivery pipeline which cause workflow and build issues.

I’ve found that the best approach is to build automation incrementally into your pipeline, building confidence as you go. If your developers are confident in the testing process, and get valuable feedback in initial stages, it will be easier to have their support and buy-in as the process continues to scale.

A Turnkey Triage Process

When it comes to managing bugs found in automation, there is a common misconception about how to triage them. People often feel as though these bugs should be treated differently than those found during manual functional testing. This is an easy way to complicate your triage process and one that I constantly see being overthought.

Keep it simple and treat your bugs found in automation like any other. The same bug found via an automated test deserves just as much attention as found by a manual tester.

A Scalable and Maintainable Solution

So much effort goes into building your automation framework that people often underestimate the amount of maintenance needed to keep it running smoothly. It’s a big lift to expand and update your framework and even test cases to handle changes to the test harness (e.g. a new browser, mobile OS version, etc.).

While applying in-house resources to this continuous process is an option, one way to easily scale this resource need is to leverage a crowd of automation testers. Such a crowd (e.g. the Applause automation crowd) can scale up and down in an elastic fashion to tackle scalability needs with respect to test case maintenance.

Regular maintenance of third-party dependencies that ease the automation process (e.g. Selenium, Appium, device cloud integrations, reporting tooling, etc.) needs to be constantly performed and tested before being used for “production” test runs. If your framework doesn’t make it VERY easy to run tests, review results, debug failures, and report status to stakeholders, you should consider investing more resources in your framework or consider allocating those resources to your revenue-generating product and evaluate commercial solutions.

As QA teams fight to keep pace with development, the quick automation fix often becomes more enticing than the sound solution. Successful automation practices are not implemented overnight though. In fact, you’re likely to spend more time and money in the long run on a quick fix than if you do things right the first time. Save yourself the hassle and implement the solution that works best for you. You’ll thank yourself later.