Most AI support projects fail in the same predictable way. A team measures deflection rate, optimises for it, and six months later has an excellent deflection number and a worse relationship with its customers. The metric was never wrong, it was just incomplete. Deflection only creates value when the customer's problem is genuinely solved without a human. Everything else is deferral.

Sort conversations by resolvability, not by volume

The usual approach is to automate the most frequent intents first. That is a reasonable heuristic for effort, but a poor one for experience. A better filter is whether the conversation can be closed with information the AI can access and actions it is allowed to take.

  • Fully automatable: the answer exists in documentation or account data, and the resolution requires no judgement, order status, invoice copies, plan limits, password and access flows, appointment changes.
  • Assist only: the answer requires interpretation or carries commercial risk, refunds outside policy, contract questions, anything involving a promise about a future date.
  • Immediate handover: the customer is distressed, the issue is a suspected outage or security incident, or the account is in an active escalation. Speed of routing matters more than accuracy of classification here.

Give the AI a real recovery path

The moment that generates lasting resentment is not the AI being wrong. It is the AI being wrong and offering no exit. Every automated flow needs three guarantees: a visible way to reach a human at any point, a hand-off that carries the full transcript so nothing has to be repeated, and a hard limit on retries. Two failed attempts at understanding an intent is the point to stop trying.

Write guardrails as policy, not as prompt text

Prompts drift. Policies persist. Encode the rules that must never be broken, refund ceilings, what data can be read out, which topics require a named human owner, as configuration in your platform, evaluated independently of the model output. That way a model upgrade cannot silently change your refund policy.

Measure containment quality, not containment rate

Replace the deflection dashboard with three numbers reported together:

  • Contained and resolved: the conversation ended with the AI and generated no follow-up within seven days.
  • Contained and abandoned: the customer stopped replying without a resolution. This is the number nobody wants to look at and the one that predicts churn.
  • Handover satisfaction: satisfaction on conversations that started with AI and ended with a human. If this is lower than your human-only baseline, your hand-off is losing context.

Roll out on a shadow week first

Before the AI replies to anyone, run it in shadow mode against live conversations and have your quality team grade what it would have sent. A week of shadow grading typically surfaces two or three policy gaps that would have become public incidents. It also produces the calibration data you need to set the confidence threshold at which the AI is allowed to answer unassisted.

Teams that follow this sequence usually end up automating a smaller share of conversations than they initially projected, and generating markedly better satisfaction on the ones they do automate. That trade is almost always worth taking.