Crowdtesting Facts: Debunking Digital Quality Myths

When people think about crowds, they think of large groups of people concentrated in one place. Typically crowds converge for a common purpose, like an arena packed for a concert or sporting event, or a shopping center around the holidays.

It’s true, crowdtesting providers assemble teams for a common purpose: finding quality defects for customers. But companies often don’t understand how crowdtesting works when developing their digital quality assurance plans. Let’s address some common crowdtesting myths to help you make better decisions about your digital quality strategy.

Myth #1: Crowdtesting replaces in-house quality teams.

Augment, expand, support, complement… Yes. Replace? No. Applause partners with our customers’ quality teams to fill gaps. Sometimes Applause adds capabilities companies don’t have in house, like developing test automation scripts or evaluating digital accessibility.

Other times, Applause helps customers ramp up for a large product launch or entry into a new geographic market. Applause can collect large volumes of data to support AI and machine learning development in just a few weeks, which enables in-house teams to focus on development rather than data collection. Offloading certain types of testing gives internal teams time to work on higher-priority projects, such as setting up test automation.

Crowdtesting provides access to platforms, networks, devices, demographics and geographic markets that labs simply can’t replicate accurately. Outside testers also bring a fresh perspective, which is hard to capture internally. “Your testing team can only do so much,” said Dewet Diener, VP of Engineering for Curve. “When you’re inside the company, you know all the warts, so you can’t go over them with a fresh mindset. This is what an external testing capability really unlocks.”

Myth #2: All software testing crowds are the same.

False again. Applause pioneered the crowdtesting model in 2007 with uTest, and has the most experience managing enterprise-scale client projects. With the largest community of testers, we can test across the most devices, mobile and wireless carriers, languages and geographies.

Different crowdtesting providers often focus on specific areas, such as functional testing, security testing or regression testing. Applause goes beyond just manual functional testing. Applause tests customer journeys, payments, user experience and acceptance. Some of our crowdtesters specialize in specific areas, like accessibility, so we can match you to a group of testers who can work on your specific needs or directly mimic your customer profile. And, Applause is the only provider able to offer clients a holistic view of digital quality across both manual and automated functional testing, with our Integrated Functional Testing solution.

Myth #3: Every tester has the same level of access to information.

Applause always protects our customer information and data:

• all testers sign an NDA with Applause

• customers can also mandate they sign a client-specific NDA; for certain pre-production projects, testers must sign an NDA before they can access details about the project

• all of our community members agree to a Code of Conduct which states that they will not talk about a client or test outside of the forum.

Each community member only has access to the projects they’re working on. The Applause community is gated, and security compliant for highly regulated industries, like financial services, media, healthcare and technology. When necessary, Applause can establish a VPN tunnel for community members to access clients’ tests, restricting access to selected participants during specified times. Once testing ends, so does tester access.

Myth #4: To get bigger incentives, testers report false positives, duplicate issues and other things that don’t matter to us.

This is another common crowdtesting myth. Applause works with each of our clients to clearly outline the scope for each test cycle. An Applause test team lead reviews all the submitted bugs to verify that they’re within the scope of work, reproducible and properly documented. Bugs that don’t meet the criteria get rejected.

Each Applause community member earns a tester rating based on their experience and the value of the bugs they submit. Those with higher ratings get higher payouts and are invited to participate in more test cycles. Our community is a meritocracy that works well for both our testers and our customers.

Myth #5: We can’t choose which testers work on our projects.

There are multiple ways to interpret this misconception. First of all, Applause offers a Bring Your Own Testers option that lets you include employees, friends and family, and other important stakeholders in your tests.

Second, customers specify tester requirements. Those criteria can range from using certain devices, operating systems and network providers to speaking specific languages, living in a particular region, or owning certain types of accounts. Applause verifies that community members meet those requirements, such as through a video that shows someone has the right accent or model car. Applause also offers uTest community members training for specific skill sets. This education can be a great way to ensure testers are prepared to meet evolving customer requirements.

Finally, customers can flag their favorite testers, typically those who deliver highly valuable bug reports, inviting them to participate in future test cycles.

The right crowdtesting partner can help companies significantly improve digital quality. The Applause Product Excellence Platform goes beyond crowdtesting: get access to the world’s largest community of digital experts, as well as world-class software products, an enterprise-grade SaaS testing infrastructure, and a wide range of digital testing solutions. Learn more about how Applause prioritizes exceptional digital experiences for our customers’ end users.