Why AI-Led Validation Is the Antidote to Mediocrity

Generative models and agentic workflows are turbo-charging software velocity. And traditional quality assurance models are struggling to adapt to the rush of non-deterministic systems and outputs that are quickly redefining the marketplace.

The easy answers aren’t often the optimal ones. Automated evaluations accelerate speed, but relying on them in isolation risks systemized mediocrity. Achieving digital distinction means pairing AI capabilities with human discernment, domain expertise and rigorous real-world validation. Speed and quality — that’s the path to the checkered flag.

In this Q&A, Applause CEO Aatish Salvi breaks down why modern software quality requires a fresh approach. He explores the non-deterministic nature of AI, the substantial risks of unmonitored agentic deployments and why human-in-the-loop validation remains essential in an age where it’s easy to settle for ordinary instead of excellence.

How has AI broken traditional software testing?

Traditional software testing meant you clicked a button, and you knew what it was supposed to do, and then you checked whether or not you did it. With AI, you give it a prompt, and it's going to do something — and that something is going to be different depending on the exact words you used in the prompt and how it's feeling at that very minute. It might be different depending on which language that prompt is in. So AI testing is non-deterministic. It does not have a defined output. So, now you really do need to understand and use AI itself, along with humans, to test AI.

Why can't companies simply automate their way to quality with today's AI tools? Well, they can automate their way to mediocrity. That's very plausible. AI is terrific at producing something very mediocre very quickly, and if you use a foundation model out of the box to start testing all of your code and all of your applications, that's what you'd get — very mediocre evaluation very fast. The way the companies are going to differentiate themselves in the age of AI is by being for the consumer that cut above. The difference between a product that is mediocre and one that is great is going to be whether you have that layer of human plus AI to enable it to cross the threshold.

Why are human insights becoming more valuable, not less, in the AI era

The biggest myth about AI testing is you can test an AI using an AI. Well, it turns out that asking an AI to test an AI is like asking your microwave to test your toaster. At the end of the day, they're two machines who don't really understand your business necessarily or your product. They're making their best guess — gosh, they're trying hard — but they don't have the intelligence and the deep domain knowledge of your legal requirements, your compliance requirements, the parts of your product that are important to you, your user flows, all the context that our testers have, that humans have. When you put that along with the AI, you get something truly incredible.

What are enterprises getting wrong about AI and agentic AI?

Enterprises believe that they have to just deploy agents as fast as humanly possible. The analogy I use is, would you let a Labrador puppy that's untrained loose in your house? I wouldn't do the same with your agents either. At the end of the day, these agents are now running around doing stuff, and there's no evaluation. There's no one actually figuring out if what they're doing is good or bad.

There's a tendency in human beings — once you've got someone doing something for you, you're just inclined to believe that it's right because it's one thing less on your plate. So they have even less human oversight than they used to before. Gosh only knows what they're doing out there.

What new quality challenges does AI create for companies?

There are the challenges it's creating while it's helping your engineers make code, which is to say that engineers are now producing code much, much faster with sometimes fewer specifications to inform them. There are fewer developers that are actually looking over that code, and more of it is shipping per release. So AI is creating a challenge where you have the same defect rate but you're producing more, so the number of defects you're actually creating has gone up in volume. So, you need to get faster and better about catching them. That's one.

The second-most important one is that AI is now all over your product. It's in your product and interfacing with your consumer. Your AI is fundamentally a non-deterministic system. It's going to behave differently every time someone gives it a prompt, depending on how that prompt was written and who the consumer was. It's going to behave differently depending on the language that prompt was written in. So, now being able to test your product is not just, "I click this button. Did it do what I expect?" You need red teaming, model evaluation — these new and modern approaches to testing AI and product, which is something we've been working hard on developing at Applause.

If we wanted to just be fast, we'd just point a foundational model out the box at your code and give you the output. It'd be very fast, and very mediocre. Instead, the choice that we're making is that we're going to take that foundational model and invest a great deal of our time and energy in training it to be incredibly good at testing your business, your products, your way.

As AI tools get more powerful, where do humans still play an important role in testing and training those systems

Humans play an important role because your AI can do work, but knowing what work to do and whether it's been done well, that is still something that humans do incredibly well. People are very good at guiding the AI towards the things that matter, at looking at whether the AI has done good work and giving it a grade. And if you do that, the AI itself gets better and more responsive, but you need that human.

You might say, "Well, maybe if you train the AI once, you're done," but that's not the way the world works. The environment around you keeps changing. Your business shifts. Your strategy shifts. Your consumers shift. The AI's ability to keep up with all of those shifts without a human helping it understand how to navigate them is very limited.

Why is a managed testing service becoming more valuable, not less, in the AI era?

Managed testing services allow you to have people who are going to take the time to deeply learn your business, your product, your consumers, your legal and compliance risk tolerances, and to use all that information to make the AIs that are testing your systems that much more capable. It can do phenomenal things, but it needs to be trained. Someone needs to actually help it understand what its real job is, help it understand when it's doing a good and bad job, help it figure out when it needs to change the job that it's doing, and point it at the jobs that are important. If you don't have that happening, then the AI is just going to try and take its best guess at what matters — in our experiments, that guess isn't all that great.

You need to have a partner that is able to take care of doing these complex evaluations, red teaming and model evaluation. They require specialized skill sets and infrastructure to conduct. So unless you're willing to build out all of that for yourself, [leveraging] it from somebody like Applause, who builds it for large numbers of companies, seems like the better bet.

What does great digital quality look like, and how has that changed recently?

I think great digital quality looks distinctive. It looks and feels a cut above the rest.

AI has the potential to enter us into an age of extreme mediocrity, where everything looks and sounds more or less the same. It's not bad, but it's not great. What makes great digital quality, I think, is something that has an exceptional feel, that still feels distinctive, like the creativity and particular viewpoint of a human has enhanced the AI's output.

When we evaluate, we want to do the same thing. We don't just want to take a model out of the box and point it at the evaluation. We want humans that have discernment, judgment and taste, who know your product line, who understand your business, who use your product every day and have their own feelings about it, to give the feedback on whether or not we've made the mark. That's going to give us distinctive products in the world. That's the world I'd rather see.

Elevate to Exceptional

Relying on automated evaluations alone risks settling for digital mediocrity. True distinction comes from moving beyond "good enough" software to deliver experiences that consistently stand out. And our goal at Applause is to help brands achieve that high standard.

By combining advanced AI capabilities with authentic human discernment, domain expertise and real-world validation, enterprise teams can achieve rapid release velocity and exceptional quality. The business impact of the Applause approach is proven. According to independent IDC research, organizations that leverage Applause services capture $1.54 million in avoided costs by resolving critical bugs. Teaming up with Applause as a true digital quality partner empowers organizations to build 70% more efficient testing teams while delivering over three times as many releases per testing FTE.

Speak with our team of experts today to align on your quality goals. True AI-driven innovation requires AI-enhanced, human-validated quality. And we’re here to provide it.