Improving Digital Quality with Community-Based Test Automation

Quality engineering and software development have evolved a great deal over the last several decades. Testing starts earlier in the development process – but quality often still lags behind development. So how do you increase the speed of quality to keep pace with shorter timelines and faster release cycles? Through strategic test automation at critical points in the SDLC, according to Adonis Celestine’s recent webinar.

The evolution of software development and testing methodologies

With the waterfall development methodology, once the standard, quality was always reactive. It took a long time for an iteration of the product to reach a stage where testing was possible. To reduce this time to market, Agile and DevOps methodologies came into picture, with a shift to more proactive quality assurance, where QA occurs while the product is being made within the sprints in the Agile and DevOps teams. With advancements in AI, machine learning and supervised learning, the industry is on the verge of another shift – this time to more predictive quality assurance, where quality starts even before the product is made.  

Speed is driving this innovation. As companies look for the competitive advantages that go to early innovators and first movers, there is more pressure than ever on developers and testers to deliver more features in less time, often at the expense of quality checks. As a result, skill sets have evolved as well. While organizations once sought workers with specialized “I” shaped skill sets, today they favor professionals with broader “comb” shaped skill sets to meet the demands of modern development.

Unfortunately, the market is experiencing a severe shortage of IT talent, making it an even greater challenge to ensure timely, effective quality assurance. To address this issue, organizations have embraced methodologies such as test-driven development, behavior-driven development and Agile, which enable more efficient and effective testing processes. In addition, artificial intelligence and machine learning are increasingly helping teams enhance testing processes. AI-powered tools can analyze large volumes of test data, identify patterns, and predict potential defects, enabling proactive QA.

The role and rise of test automation

All these evolutions were efforts to ensure that quality assurance happens at the speed at which the code is being developed. Automation plays a crucial role in streamlining testing processes, reducing manual effort, and improving overall efficiency. The adoption of DevOps practices has facilitated closer collaboration between developers and testers, enabling continuous integration and continuous delivery of software. 

The tools that support automation have evolved as well, including low-code and no-code tools that promised to democratize automation. But in reality, low-code, no-code platforms often come with a steep learning curve. Maintenance or custom solutions required for a particular application still require some specialized expertise. And with the recent advancements in AI, organizations have access to AI-driven test case management, AI-driven test design, AI-driven test automation, AI-driven continuous delivery. And yet, speed remains a challenge: quality often fails to keep pace with the speed of development. 

“We managed to move from DevOps to DevOops, where we are releasing stuff to production much faster, but not with the right quality,” said Celestine. He went on to point out that tools and methodologies don’t solve problems: people do. But with an ongoing shortage of IT talent, many organizations don’t know how to find people with the right expertise to help them achieve speed and scale. “How do you get the right test automation coverage within a sprint? How can you test everything that you want to test? The answer is, you can scale your automation with community,” he explained.

How community automation experts can support internal teams

Accessing resources from a crowdtesting community allows companies to tap into a group of people with automation expertise to help an in-house team solve automation and coverage problems. “Imagine, you have a large pool of resources who you can use whenever you want to solve a specific problem or on demand whenever you have a peak in your development efforts. You just make use of them, solve the problem, and then you are done, or you can use them in the next upcoming spike. So that’s the flexibility that crowd-driven automation provides,” said Celestine. 

The community consists of test automation engineers from around the world, with smaller groups or subsets with particular skills, such as Java automation, JavaScript automation or Python automation. Community members are chosen based on their technical skills. Celestine said, “They form what we call a vetted crowd-driven automation group, which we leverage to solve this problem of capacity and speed and reduce the time to market for our customers.”

Steps for getting started with community-based automation

Celestine outlined several steps for shifting from a highly manual testing team into a fully automated enterprise. 

1. Have a vision and executive sponsorship. Understand your goals and what you want to automate from a quality assurance perspective. 
2. Decide what can be automated and what cannot be automated. Celestine pointed out that not all tests are good candidates for automation, despite the DevOps focus on automating everything. “The moment you do that, you end up with a large suite of test cases that cannot be maintained properly, that does not provide a return on investment. It’s important to define what to automate and what not to automate, especially the human aspects like usability, accessibility, customer experience.”
3. Start with a proof of concept. While Celestine emphasized that tools and methodologies alone do help organizations gain the right amount of speed, choosing a wrong tool or wrong methodology can also have bad effects on your overall development methodology. “Do some proof of concepts to see what would be the right technology sets and the right tool sets that you want to use to drive this crowdsourced automation initiative.” As with any automation project, community comes with an investment and return on investment calculation, so Celestine encourages teams to think about the investment cost upfront.
4. Engage with a crowdtesting partner’s automation community. “Having the partner with the right amount of skills in the community, with the right amount of vetted candidates and security procedures, that’s also very important from a vision and governance perspective,” Celestine said. With Applause’s community, members are tested for technical skills, he explained. In addition, a project manager checks all the pull requests that are being done by the automation engineers from the community, approves them, allocates an estimate to those tasks, and manages pay out based on the tasks that are completed. 

Celestine ended the webinar with a concise wrap-up: “To summarize, there are three things I want you to take away from this webinar. One, speed is really important for business, but we do not do enough or the right level of quality coverage to release things confidently in production. Two, crowdsourced automation can be a way to solve this problem to infinitely scale your capacity and bring quality and speed to your quality assurance process. Third, even though crowdsourcing is quite good in solving your problem, approach it with care. Ensure that there are right security and confidentiality principles in place.”

Webinars