Plan requirement
| Subscription | Suite Team or higher, Support Team or higher |
| Access | Admin |
Testing means trying to break it, not confirming it works. Four ways to do that before customers do.
Test it
- Open the AI agent and click Test AI agent. The test widget opens on the welcome reply, with every setting the agent currently has, instructions included.
- Work through your list of questions.
- Note every answer that is wrong, vague, or arrives after too much back and forth.
- Fix the cause, usually in an article, and test again.
Two things make this safe to do as often as you like. Test conversations do not count towards your automated resolutions and stay out of the Explore reporting, and they are still recorded in the conversation logs, so you can reread one you closed too quickly.
While you are building a single flow, Test dialogue in the dialogue builder runs just that one, and the test icon on a block runs the branch from there. Use those while writing, and the test widget to judge the whole thing.
One: ask badly
Short, misspelled, without context. "order late", "cant login", "wheres my stuff". That is what real messages look like, and an agent that only handles well-formed questions handles almost nothing.
Two: ask what it should refuse
Something you have not documented. Something outside your business entirely. It should say it cannot help and offer a person, not improvise.
An agent that answers everything is an agent that will eventually answer wrongly with confidence.
Three: change subject mid-conversation
Ask about an order, then about returns, then back. Real conversations wander, and this is where an agent either keeps up or loops.
Four: ask for a human
Directly, and also indirectly by getting frustrated. Handover must work from any point, and it should not require the customer to phrase it a particular way.
Get somebody else to try
You know what the agent can do, so you unconsciously ask in ways that work. A colleague from another team will find in ten minutes what you would not find in an hour.
Then test it where customers are
The test widget cannot tell you whether your actions, triggers and handovers behave in your own environment. Connect the agent to Support and embed the widget in a sandbox or staging copy of your site or help centre, and run the same list there. That is the version of the test that finds the integration problems.
Keep the list
The questions you test with become your regression set. Run them again after every substantive change, because a fix in one place moves behaviour in another.
Comments
0 comments
Article is closed for comments.