Four Things Slowing Down Your Automation Practices
Test automation has hit a fever pitch recently—and for good reason. As companies build towards an agile testing environment, they are finding automation necessary to achieve the level of code coverage they desire (and need) to provide an optimal user experience.
Though the solution is apparent, implementing a successful automation strategy is not. Too many companies make the investment in automation, only to see it fail shortly thereafter. Here are four reasons companies are failing in their automation efforts and what you need to think about when planning.
Picking the Wrong Tools for the Job
The first important question you need to ask yourself is whether you want to build or buy your automation framework. It’s not that black and white though. Each option has its pros and cons, but it ultimately comes down to what works best for your company.
Are you a smaller team that is just digging its teeth into automation? Getting started with an off-the-shelf enterprise solution may be the best avenue for you to take. This solution makes it easier to find a qualified test engineer to get you up and running. It also leaves you with some confidence that you’re working with time-tested tools while also having the support to handle any bugs along the way.
For a team with a bit more functional maturity, open-source automation may be more your flavor. It provides the customization that a seasoned team will value and provides more long-term adaptability as your automation suite grows. Along with that, open-source frameworks are free to use but often require more technical expertise to implement and customize to your liking.
Other teams may choose to go the hybrid route which has benefits of its own. Ultimately, the tools you choose for your automation practices should be thoroughly considered and vetted by your entire team. If you choose incorrectly, you could set yourself back in both time and money.
Doing Too Much, Too Fast
Once you establish your automation framework, the immediate instinct is often to go all in and automate a significant portion of your deployment pipeline. Over-engineering at the outset will delay any initial value that you could realize from automation. Not only is this extremely ambitious, but it will likely cause more problems than it provides benefits.
Automation should not be integrated directly into your development pipeline. In doing so, you are likely to cause workflow and build issues as you work out any kinks in your test cases. When getting started, you will want to run your automation tests in parallel until you’re seeing consistent results. Once you’re confident in the results, you can begin to incrementally build those test cases directly into your pipeline.
I recently spoke with a client about their failed initial automation effort. After spending some time with the engineering team, their issue was glaringly apparent. They had started out by immediately integrating UI test automation results directly into their pipeline after unit tests were complete and the build was deployed to an integration environment. They coupled the deployment and UI tests into a single job, blocking the availability of the environment until tests were complete and (even worse) failing the deployment if the UI tests failed.
While the fix was easy to get them back on track (e.g. moving UI tests to an upstream, stable environment as they matured their process and decoupling tests from deployments) the constant interruption of the developer’s workflow caused them to lose confidence in the initiative and eventually halt it.
Undefined Triage Process
Automation without a defined triage process is a fruitless endeavor. On countless occasions, I have seen companies spin their wheels because they fail to provide consistent documentation of their bugs. In order to speed up your triage process, your dev team needs easy access to logs, screenshots, and test result assets. Anything less will not suffice.
Keep in mind, your automation test results should be triaged just like bugs. With automated tests running on a pass/fail basis, it’s common for people to kick back a failed test without looking further into the issue. However, these issues are often a result of a test script needing to be updated or an issue with the environment being tested in, rather than the code itself. This highlights the need for an intermediary stage we call triage, similar to when triaging bugs.
Poorly Maintained Framework and Test Cases
Automation is meant to make things easier and faster for development teams, but without proper maintenance, teams will experience just the opposite. Test cases aren’t just a one-and-done situation, but instead need to be continually updated to address non-trivial UI changes, major OS/browser changes, automation library changes, and more.
It’s also critical that maintenance is taken into account during framework construction. Complex design patterns can lead to difficult, timely, and cost-prohibitive maintenance efforts (e.g. locator strategies, page factories, etc.) that will derail your automation efforts. This is especially important when you factor in employee turnover and the need for new developers to quickly adapt to a new framework.
Without automation, a company can never truly achieve agile development. With that said, a company exhibiting poor automation practices will also never achieve agile. It’s not an easy or quick process, but time spent doing things right the first time will pay dividends for you in the long run. Crowdtesting can help you get there.
