Your Support Metrics Break When AI Takes Over. Here Is the Fix.

The 10% Blind Spot

CSAT captures less than 10% of your conversations. The responses you do get skew toward extremes: the delighted and the furious. The vast majority of customers say nothing at all, especially when they got what they needed and wanted to move on.

That silence is not harmless. You are reporting to leadership and coaching your team based on a sample that does not represent most of your customers. No amount of survey design fixes a response rate that low. CSAT also compresses multiple problems into a single score. A negative rating could come from product frustration, a policy dispute, or genuinely bad service, but the score alone will not tell you which. You end up spending more time debating the data than acting on it.

When an AI agent handles 80% of conversations end-to-end, the gap gets bigger. More of your customer experience sits outside direct human review. The conversations the agent resolved cleanly, the ones where the customer got their answer and left, are the ones you cannot learn from, because you never asked and they never told you. Meanwhile, the feedback you do get comes from the people who had a strong enough reaction to fill out a survey, which is not a representative sample.

What Full Coverage Actually Looks Like

AI flips this problem entirely. Instead of waiting for customers to tell you how it went, you can evaluate every interaction automatically. This is the shift from sampling to census-level data, and it changes what you can know about your product.

Intercom's Fin team built CX Score for exactly this purpose. It scores every conversation (both AI and human) on a 1-5 scale, surfaces the reason behind each score, and gives roughly five times more conversation coverage than CSAT alone. The reasons matter more than the number: instead of guessing why a score is low, you see whether the problem was answer quality, customer effort, or a product gap.

The principle transfers to any AI agent. If you are deploying one, you need visibility into every conversation, not just the ones where someone clicked a star. The models that handle your conversations can also evaluate them. The question is whether you set up the feedback loop or leave it to chance.

The Salesforce acquisition of Fin for $3.6B in June 2026 is a signal that AI agent customer experience is a real category, not an experiment. Companies are willing to pay significant premiums for the infrastructure to get this right.

Setting Targets from Real Data

You cannot map old CSAT targets onto a new metric. The coverage is fundamentally different, so you need to build targets from what the data actually shows.

The Fin team started by correlating their CX Score with operational metrics like first response time and time to close. This gave them useful baselines for human support. Then they decomposed the score: answer quality, customer effort, product feedback. Answer quality had the biggest impact on the overall score, which makes sense when the agent handles the majority of conversations. That told them where to focus.

At roughly 80% automation rate, they modelled what the score would look like if they eliminated low answer quality across both Fin and human conversations. They set initial targets: 80% for Fin, 70% for human support, 78% overall. They have since raised those targets as performance improved.

The key insight: build targets from your actual data, not from thresholds inherited from a metric that measured something different. CSAT at 85% on a 10% response rate is not the same as CX Score at 78% on 50% coverage. The number means something different, so the target means something different.

Closing the Loop

When every conversation is scored and the reasons are visible, recurring problems become traceable. You can identify which topics and conversation types are scoring poorly and why. You can see how scores differ across channels and between agent and human conversations. You can trace which operational issues are creating friction for customers.

Instead of working from a handful of survey responses and caveating how representative the data is, you route issues to the right owner. You fix them at the source. You track whether the fix worked. And you prevent the same problem from affecting the next customer.

This is the operational loop most teams deploying AI agents are missing. They measure resolution rate and CSAT and call it done. They do not look at what the agent actually said, whether the customer had to repeat themselves, or whether a technically correct answer still felt frustrating because it missed the context of their actual use case.

For example, a manager might see that a particular topic is underperforming across the team and use that to update knowledge base content or run a focused session on how that topic should be handled. Each pattern leads to a specific action, instead of a vague signal that something might be off.

What This Means for Builders

If you are building or deploying an AI agent right now, here is where to start:

Audit your CSAT response rate. If it is below 20%, you have a coverage problem that no amount of metric-tuning will fix. The sample is too small and too biased to act on.

Build or buy conversation scoring that covers every interaction, not a sample. The AI that handles your conversations can also evaluate them. This does not require a separate analytics platform. It requires a feedback loop from the conversation to a quality model.

Break quality down by attributes. An aggregate score hides more than it reveals. You need to know whether the problem is answer quality, customer effort, or product gaps, because each one has a different fix.

Set targets from your actual data, not from old benchmarks. The distribution of scores when you evaluate every conversation is fundamentally different from survey responses. Start by correlating your new metric with existing operational data, model what good looks like, then set a target you can raise.

Close the loop. Every negative score should trigger a specific action: update knowledge base content, retrain on a topic, fix a workflow gap, escalate to product. If nobody owns the follow-through, the measurement is theatre.

Teams that get this right will have a compounding advantage. Their agents improve faster because they see what is actually wrong, not just what a handful of vocal customers complained about. In a market where AI agent quality is becoming a differentiator, that advantage compounds every week.