Containment rate

What containment rate actually measures

Containment rate counts conversations the AI resolves from first message to a verified outcome without a human agent intervening. The usual formula is contained contacts divided by AI-eligible contacts in a fixed window, times one hundred. Eligible means the bot was allowed to attempt the intent, not every spam email or VIP queue ticket.

A thread where the bot answers a policy question but an agent still issues the refund, edits the reply, or merges the ticket is not contained. Helpdesk status "solved" alone is not proof the shopper got what they needed. I define AI-only resolution against concrete ecommerce outcomes: tracking explained with live carrier data, return label emailed, exchange eligibility confirmed, cancellation completed, or a documented policy answer that needs no store action.

If your tags cannot name those outcomes, the percentage is theater. Tie resolution tags to Shopify or WooCommerce order events so the metric reflects real state changes, not chat auto-close. Write the definition on a one-pager: good example, bad example, and when the bot must stop.

AI-only resolution is not deflection

Deflection rate measures how much support demand never needs a human reply. That can include a help article, a proactive delay email, an order-status page, or a bot chat that ends without escalation. Containment is narrower: an AI session that actually finishes the work. A shopper who reads a returns FAQ and never writes in was deflected; they were not contained by your AI agent.

A shopper who chats with the bot, receives a prepaid label, and needs no human was both deflected and contained, depending on how you frame the funnel. Vendors blur the two because deflection is easier to inflate. Abandoned chats, "thanks" macros, and article redirects look like wins on a deflection slide. Containment should require a resolution taxonomy and, ideally, a linked order event.

Read ticket deflection for demand reduction practice, and keep deflection rate as the broader KPI. In weekly ops I show both: deflection for volume shape, containment for whether the AI is closing real work.

Pair containment with CSAT and QA on contained tickets

Containment can climb for the wrong reasons: aggressive auto-close, vague answers, or a hidden "talk to a person" path that trains customers to give up. Volume alone never tells you if the AI met the standard you expect from a trained agent. I always report CSAT or thumbs-up rate on the contained cohort only, not blended with human tickets.

A rising containment rate with falling contained CSAT means the bot is trading experience for a dashboard win. Run the same QA rubric on AI closures that you run on agent work: correct policy, accurate order facts, right tone, and no invented tracking or refund timelines. Sample weekly and score a fixed batch per intent so WISMO does not hide a broken returns flow.

Zendesk's materials on AI agents help when you evaluate enterprise automated-resolution claims, but your rubric still lives in helpdesk tags. If reviewers disagree on whether a closure was correct, freeze that intent until knowledge and tools are fixed.

Reopen rate is the containment reality check

Reopen rate answers the question containment cannot: did the "resolved" issue stay resolved? Track same-issue recontact within a fixed window, commonly seven days, keyed by order ID plus intent, not only by ticket ID. If the bot says a package is on time and the shopper emails two days later because it is lost, that first ticket was not durable containment.

Count the second contact as rework and reverse the original containment credit in your ops report, even if the vendor dashboard still celebrates the first close. Ecommerce reopens cluster around late deliveries, partial refunds, and exchange inventory misses. Instrument reopens with a tag such as `reopen_after_ai` and a reason code so you can see which intents are lying to you.

A healthy program shows stable or rising containment with flat or falling reopen and stable CSAT. Rising containment with rising reopen is a silent quality failure. I would rather force an early agent handoff with full order context than protect a containment target that manufactures follow-up email.

Common mistakes: bonuses and aggressive auto-close

The fastest way to corrupt containment is to bonus staff or vendors on the rate alone. People will hide the escalate button, shorten idle timeouts, and teach the bot to say "your order is on its way" without checking the carrier. Aggressive auto-close is the second trap: the system marks solved after one FAQ snippet even when the shopper asked for a refund exception.

That improves containment for a day and floods reopen and social DMs the next week. Never let solved status equal contained without an outcome tag and, for action intents, an order-event check.

Other failure modes I see in pilots: counting human-edited AI drafts as contained; excluding email and social so only chat inflates the rate; expanding write tools before read-only WISMO is stable; and comparing your number to a vendor benchmark with a looser definition. Use the AI support ROI calculator only after category-level containment, CSAT, and reopen. Not after a sales-deck headline. Optimize for durable resolution and clean handoffs.

Containment is a diagnostic, not a trophy.