Conversation Design Is the Missing AI Agent Discipline

If nobody on your team has trained your AI agent on how to communicate, it defaults to sounding like an LLM. Because it is one.

That matters because customers can tell. They may not say "this agent needs conversation design." They just feel it. Responses that are technically correct but strangely structured. Too much detail when a short answer would do. A flat tone answering a frustrated message. A silent handover to a human where they have to repeat everything.

Intercom's Fin team ran an A/B test that makes the impact concrete. They tested two opening messages. The old version: "Hi, you're speaking with Fin AI Agent. I'm here to answer your questions. You can always talk to the team if you need to. How can I help?" The new version was warmer and more conversational, setting expectations and inviting detail. That single change lifted CSAT from 72.8% to 78.4%.

No model upgrade. No training data overhaul. Just the first thing the customer sees, said differently.

The discipline behind that kind of improvement is conversation design. Most teams building agent experiences have never heard of it, let alone assigned ownership for it. Here is what it covers, why your CSAT is probably lying to you, and how to start tomorrow.

The measurement problem that hides the gap

Before you fix how your agent communicates, you need to know whether it is working. Most teams rely on CSAT surveys. And CSAT has a blind spot the size of your agent's conversation volume.

Intercom's data shows that surveys capture less than 10% of conversations. The responses you do get skew hard toward the extremes: the furious and the delighted. The vast majority of customers say nothing. They are busy. They move on.

That silence is a blind spot. You are reporting to leadership and making product decisions based on a sample that does not represent most of your customers. As AI handles more conversations end to end, the gap gets bigger. A larger share of your customer experience sits outside direct human review, while CSAT still only reflects the conversations where someone chose to fill in a survey.

The fix is automated evaluation of every conversation. When every interaction is scored and the reasons behind each score are surfaced, recurring problems become traceable. You can identify which issues drive negative scores, how often they happen, and whether the root cause is support, product, or a specific workflow. This is the foundation that conversation design builds on. You cannot improve what you cannot see.

The five areas of conversation design

Once you have visibility, the next question is what to optimize. Intercom's conversation design framework breaks it into five areas:

Tone and personality. Is your agent formal or casual? Does it adjust when the customer is frustrated versus just browsing? Most agents pick a single register and stay there, which means they sound wrong in at least some interactions.

Response structure. Does the agent match the level of detail to what was asked? Left unguided, an LLM will happily deliver a three-paragraph explanation when the customer just wanted a yes or no. Over-explaining is one of the most common conversation design mistakes.

Handoff logic. When should the agent escalate? How does it communicate the transition? What context does the human rep receive? The handoff is one of the highest friction moments in any agent conversation. A silent transfer where the customer has to repeat themselves can undo everything the agent accomplished.

Interaction flow. How does a conversation progress through question, answer, resolution, or escalation? Does the agent ask clarifying questions, or guess and hope? Good flow design anticipates the paths a conversation can take and handles each one cleanly.

Response quality. Is the answer clear, helpful, and on-brand even when it is technically correct? This is the difference between "Your subscription will be upgraded within 24 hours" and "We have received your upgrade request and your new features will be available by this time tomorrow. You will get a confirmation email when it is done."

Where most teams mess it up

The most common mistake is treating conversation design as a one-time setup. You write a system prompt, set some guardrails, and assume the agent will figure out the rest. It will not. Every new feature launch, every product change, every workflow update is a conversation design change.

This is where the concept of agent readiness becomes critical. When the product team ships a new feature and customer questions start landing, the agent needs the information to answer them. If it does not have it, every one of those conversations gets handed off to a human. The team gets flooded right when volume peaks. Customers lose trust in the agent. The support team loses the capacity it was supposed to gain.

The fix is making agent readiness part of the launch process. A feature should only be considered shipped when the agent can answer questions about it correctly. That means getting someone from the support side into product kickoff meetings, testing new features before they launch, updating knowledge base content, retiring old content that conflicts with new behavior, and running test queries against the agent before customers start asking.

How to start without a dedicated hire

Conversation design sounds like a role you need to hire for. You may eventually. But there is a lot you can do right now with the team you have.

Write down how your agent should sound. One paragraph. Not a brand book. Just a reference: direct and helpful, or warm and conversational, or whatever fits your product. Come back to it when you make tone decisions.

Design the handoff like it matters. The customer should never have to repeat themselves. The human rep should receive the full conversation, what the agent already tried, and why the handoff happened. Communicate the transition clearly instead of going silent.

Know when the agent should stop talking. Over-explaining is the number one conversation design mistake. If the customer asks a yes/no question, give a yes/no answer. Add more context only when they ask for it.

Build a follow-up process. If someone drops off mid-conversation, reach back out. Most teams ignore this. Customers notice when no one follows up. A simple automated check-in after 24 hours can turn a dead end into a resolved interaction.

Make agent readiness part of every launch. Before you ship your next feature, ask: can our agent answer three realistic questions about it correctly? If not, do not ship. That simple gate will force the discipline into your process.

The payoff

The numbers from Intercom are not an outlier. A warmer greeting lifted CSAT by 5.6 points. An A/B test on a single interaction point moved a metric that most teams treat as a lagging indicator of everything.

The gap between "the LLM decides how to talk" and "your team decides how to talk" is the difference between a commodity agent experience and one that customers actually trust. As more products ship AI agents, conversation design will be the differentiator that separates the agents customers remember from the ones they try to bypass.

The only wrong move is not assigning ownership for it. Someone on your team needs to own how the agent sounds. If nobody does, the LLM will decide for you.