Measuring Success: CSAT, Resolution Rate, and Cost per Ticket
The right CS AI metrics combine efficiency (containment rate, handle time, cost per ticket) with quality (CSAT, resolution rate, repeat contact rate). Optimizing efficiency in isolation almost always tanks quality.
- ·Define and measure the key KPIs for AI customer service performance
- ·Build a feedback loop to improve AI performance over time
- ·Identify when AI customer service is not working and how to course-correct
Measuring Success: CSAT, Resolution Rate, and Cost per Ticket
What gets measured gets improved. Without clear metrics, you cannot tell if your AI customer service investment is working.
The Core Metrics Framework
**Resolution rate (most important):**
Percentage of tickets fully resolved by AI without human intervention.
- ›Target: 30-60% for chatbots, 60-80% for well-tuned AI agents
- ›Measure: tickets closed without any agent touch / total tickets started with AI
**Customer Satisfaction (CSAT):**
Customer ratings of their support experience. Track separately for:
- ›AI-only interactions (target: 70-85% positive)
- ›AI + human escalation (often higher than pure AI — 80-90%)
- ›Human-only interactions (your baseline)
**First Contact Resolution (FCR):**
Was the issue resolved in the first interaction, without follow-ups?
- ›AI typically improves FCR for simple issues
- ›FCR should be tracked per issue category, not just overall
**Cost per ticket:**
Total support cost / total tickets resolved. AI should reduce this.
- ›Track human-handled vs AI-handled cost per ticket separately
- ›Fully AI-resolved tickets should cost 10-25% of human-handled tickets
**Time to First Response:**
How quickly does the customer receive a substantive response?
- ›AI should bring this to under 1 minute for chat (vs minutes to hours for human queue)
- ›Critical for customer experience: 90% of customers rate fast response as "important" or "very important"
Building a Feedback Loop
AI customer service improves only if you actively train and improve it:
- ›Review 20-30 escalated conversations per week — what should the AI have handled?
- ›Review 20-30 AI-resolved conversations — were they actually resolved well, or did the customer return?
- ›Update knowledge base when new product information changes AI answers
- ›Tune escalation triggers based on false positives (unnecessary escalations) and false negatives (should-have-escalated)
Red Flags That AI Is Failing
- ›Containment rate dropping below 30% (customers constantly escalating)
- ›AI CSAT lower than human CSAT by more than 15 points
- ›High rate of customers returning to support within 7 days with the same issue
- ›Agents reporting the same AI mistakes repeatedly (systemic, not one-off)
Key Insights
- Resolution rate (% resolved without human) is the primary AI customer service metric — target 30-60%
- Track AI CSAT separately from human CSAT — if AI CSAT drops 15+ points below human, fix before scaling
- Cost per ticket: fully AI-resolved tickets should cost 10-25% of human-handled tickets
- Build a weekly feedback loop: review 20-30 escalations and 20-30 AI resolutions to find improvement patterns
- Red flags: containment rate below 30%, customers returning with same issue within 7 days, repeated agent complaints
Why It Matters
Many CS teams declare AI victory based on containment rate alone — the percentage of tickets resolved without a human. This is a trap: high containment with poor quality just means you stopped the customer from reaching help. Measuring resolution and repeat-contact alongside containment exposes the trade-off and forces honest decisions about where AI is actually serving customers vs. just deflecting them.