Plan requirement
| Subscription | Any plan |
| Access | Admin |
Four numbers tell you whether an AI agent is working. Two more get quoted and mean less than they appear to.
The four
- Share of conversations resolved. Not the count, the share, so it survives changes in volume.
- Handover rate, and how far into the conversation it happens. Early handovers are healthy; late ones are wasted customer time.
- Repeat contact after a resolution. Somebody coming back within a day on the same subject was not helped, whatever the resolution count says.
- Ticket volume on automated subjects. Whether the questions the agent handles are actually leaving the queue.
The two that mislead
Total conversations. It rises with traffic and says nothing about quality.
Automated resolutions as a raw count. Useful for billing, weak as a measure: a busy month looks like an improvement.
Read them together
A high resolution rate with high repeat contact means customers are giving up, not being helped. Either number alone tells the wrong story, and the pair is unambiguous.
Compare against before
Whatever the agent handles, compare against the same period before it existed. That comparison is what answers the question the business is actually asking.
Read transcripts alongside
Numbers say something changed; transcripts say what. Ten conversations a week is enough to keep the numbers interpretable, and it is the habit that keeps a deployment improving.
Comments
0 comments
Article is closed for comments.