From Drift to Deflection: Engineering Trust in AI Systems

Trust in AI is a deliberate engineering outcome achieved through three core pillars: rigorous evaluation, proactive observation, and adversarial testing. In this blog post, I will explain how we can help our customers prevent their AI agents from drifting and move to higher deflection rates.

Trust is engineered

Imagine you are taking an intercontinental flight. You board the plane, make yourself comfortable, and arrive at your destination, safe and sound. Despite the fact that you are flying 35,000 feet above the ground at 500 mph in a metal tube, most of us feel safe enough to relax or even sleep. Millions of passengers place their trust in airlines every single day. Without this trust, no one would take a flight. 

The trust we place in air travel is the result of meticulous engineering, redundant safety systems, and the rigorous training pilots and cabin crew undergo. In aviation, trust is engineered before the wheels ever leave the runway.

The same must be true for AI-driven self-service. For a customer to trust a chatbot with their billing issue or booking request, that system must be built on a foundation of precision engineering and extensive testing. Now, as autonomous AI agents begin to take on more complex tasks, this trust becomes the primary success factor. For these agents to succeed, they must achieve high deflection rates without sacrificing the customer experience.

What is a deflection rate?

Deflection rate is a key performance indicator that measures the percentage of customer support inquiries successfully resolved through automated self-service channels without human intervention. Self-service channels can include AI chatbots, virtual assistants, IVR systems, or knowledge bases.

Calculation: (Issues resolved via self-service ÷ Total issues submitted) x 100.

Out of all the use cases for AI, customer support is perhaps the most visible. Because chatbots act as a brand’s frontline, the cost of failure is high: They can be thought of as the ‘pilots’ of your AI systems. A hallucinating bot or an unhelpful self-service process can create a customer service bottleneck, erode trust in the brand and push customers toward a competitor. High-profile instances of toxic or biased AI and security breaches are a warning that trust is hard-won but easily lost.

According to Gartner predictions, task-specific agents will be integrated in up to 40% of enterprise applications by the end of 2026, up from less than 5% today.

The three trust pillars of AI

In the scramble to deploy generative AI and reap the efficiency gains of automated support, many enterprises have neglected the most critical step in the development lifecycle. While the design may be sleek and the models trained on massive datasets, insufficient testing remains the most common point of failure. In a self-service context, testing cannot be a one-and-done task. Model drift means that pre-release testing is insufficient for long-term success.

To achieve a high deflection rate that actually sticks, enterprises must commit to a continuous cycle of pre- and post-release testing. By validating chatbot interactions against real-world human behavior, brands can help ensure their AI actually resolves real customer issues. In this way, companies can mitigate the financial and reputational risks of AI failure and build trust in self-service channels. 

To maintain a high deflection rate, enterprises must follow three trust pillars: evaluation, observation and adversarial testing.

1. Evaluate: Response grading and benchmarking

The first trust pillar involves establishing a rigorous, human-in-the-loop evaluation framework. Because LLMs are non-deterministic, traditional pass/fail automated tests are insufficient. At Applause, we advocate for prompt and response grading, where diverse testers from our global community evaluate AI outputs against a structured rubric to move beyond mere plausibility and toward better user experience. Systems are given grades in categories such as:

• Accuracy: Assess whether the information is factually correct and not just a confident hallucination. 
• Relevancy: Does the response directly address the user’s intent? High relevancy ensures the AI doesn’t just provide a correct answer, but the right answer for the context.
• Completeness: Verify that the AI provides all the necessary information to resolve the query.
• Responsiveness: Measure both the technical latency (how fast the bot responds) and the conversational agility (how well the AI adapts to follow-up questions).
• Barge-in context: Particularly in voice-enabled or multimodal channels, this category evaluates how the system handles a user interrupting it.
• Understandability: Evaluate whether the language is clear, concise, professional and appropriate for the typical user. This can be extended to evaluate tone and brand alignment. 
• Entity extraction: This category grades the AI on its ability to accurately identify and capture specific data points, such as account numbers, dates and names, and use them correctly.
• Safety & security: Verify the system is sufficiently protected against prompt injections, jailbreaking and data leaks.

When implemented correctly, AI-powered self-service can reduce customer service interactions by 40-50%. Results may differ based on the complexity of the use case, so it is always advisable to conduct benchmarking studies tailored to your specific business. Besides deflection rate, other KPIs to consider include CSAT, abandon rate and first-contact resolution.

2. Observe: Model drift monitoring 

In the traditional software world, code is typically static. But in the AI world, models are probabilistic and prone to change. Due to model drift, the performance of even the most sophisticated chatbots tends to degrade over time. Continuous monitoring is one of the ways to ensure that your AI systems are still performing as intended.

Model drift occurs when reality changes faster than your model can adapt. Despite comprehensive training and initial testing, even the best models will encounter issues as they face three types of drift:

Input drift: When the features change

Input drift occurs when the data your chatbot receives in production no longer resembles the data it was trained on. This can often happen when a new user segment is introduced. For example, if a retailer expands to a new geographic region, users might use different dialects, cultural references or slang that the chabot was not trained to understand. As a result, it can fail to recognize user intent, causing confidence scores to plummet.

Label drift: When the ground truth changes 

When definitions of ‘right’ and ‘wrong’ shift, chatbots can experience label drift. Continuing with the retail example, label drift could happen when a company changes what counts as suspicious activity or customer churn. Technically, the model might still be operating as intended; but if its responses are based on an outdated ground truth, the chatbot will come to false conclusions. 

Concept drift: When the relationship changes

Concept drift happens when the underlying relationships and patterns the model learned start to change — either gradually or suddenly. An example could be a change in buying behavior caused by seasonal demand or a sudden trend. The algorithm’s reasoning will become invalid, leading to incorrect predictions, responses and recommendations.

3. Break: Adversarial testing

Where the previous two trust pillars deal with ordinary, expected behavior from users, enterprises must also consider scenarios outside the ‘happy path’, such as attempts to manipulate the AI’s reasoning and circumvent guardrails. This is known as adversarial testing or red teaming. It aims to protect brands against users who may deliberately or unintentionally try to break AI systems.

What happens if a user tries to convince a chatbot of a refund policy that doesn’t exist? How will the chatbot respond if a user invents an elaborate story in an attempt to access private data? If the AI is not properly grounded and tested against logical manipulation, it may hallucinate an approval just to be helpful.

Automation is predictable, but humans are not. So while automation can simulate adversarial prompts at scale, it is essential that red team testing includes expert human testers. 

Safe landing: Maintaining trust in AI systems

A plane only stays on course due to constant micro-adjustments made by sophisticated systems and expert pilots. Your AI system is no different. By committing to continuous evaluation, observation and adversarial testing, you can engineer chatbots that foster customer trust. This means going beyond basic automated checks and embracing a combination of automation, human judgment and AI-powered validation.

At Applause, we can help you source the ‘flight data’ and expert crew needed to keep your AI on course. Our methodology for digital quality in AI includes:

• Golden dataset creation: We don’t rely on generic benchmarks. Domain experts and real-world reviewers curate an authoritative source of truth tailored to your specific use case, risk profile and business policies. This means your evaluation scores reflect real-world capability, not artificial lab performance.
• Scaled coverage: Using synthetic expansion techniques, we extend the benchmark to cover realistic edge cases, adversarial prompts and distribution shifts. Selected models and techniques minimize training data overlap, safeguarding against circular evaluation.
• Multi-model jury evaluation: To help remove bias and increase confidence, we utilize three or more independent frontier models in tandem when scoring outputs. By using structured rubrics and measuring inter-rater reliability, we flag the specific cases that require human intervention, providing a more defensible metric than single-model scoring.
• Expert audit loop: Our human specialists act as the final authority, reviewing model disagreements and resolving gray areas. This helps improve your dataset with every cycle, providing a clear paper trail for future risk and compliance audits.

Follow the three trust pillars: Test the agent to make it reliable; monitor the agent to keep it relevant; and challenge the agent to make it resilient. In doing so, you can verify that your AI agent is a positive and dependable representative of your brand.

Contact us to discuss how we can support your AI testing strategy.