Debugging: trace a wrong or odd answer

Lesson 7 of 11 · Debugging. You trace an odd answer from the conversation logs to its source, and add the question to your test sets from lesson 4.

Lesson 7 of 11 · Debugging. You trace an odd answer from the conversation logs to its source, and add the question to your test sets from lesson 4.

The scenario

A colleague forwards a ticket. A customer asked your AI agent a question and got an answer that is no longer true, for example a return period your policy changed months ago. Your colleague asks what went wrong and who should fix it.

Goal: find where the answer came from and reproduce it, so the right content gets fixed.

What you need

• Access to the conversation logs of your AI agent.
• The time of the customer's conversation, or the ticket.
• To open the ticket from the preview: the agent or admin role in your Zendesk account.

Steps

1. In the AI agents dashboard, go to Conversation logs and open the customer's conversation. Click Show in preview, just before Details.

   

2. The conversation opens read-only in the channel the log names. You see what the customer saw, with the buttons and sources of each answer. See Viewing a logged conversation in the preview.

   

3. Hover over the answer and click Inspect message. The Message tab of the inspector shows the Raw AI agent text, what the channel received, and Checks.

4. Open the Log tab. Under Knowledge search you see the Search question, the Reasoning and the articles the search found; the ones your AI agent used are marked cited. That tells you which article still has the old information. The Log tab is described in Reading the conversation log.

   

5. Click Open ticket in the toolbar, followed by the ticket number, to see what happened after the answer, for example whether an agent stepped in.
6. Click Close preview. Start a test with Test AI agent, choose Show as Web Widget, and send the customer's question word for word.

7. Use the test tools next to the messages. On the customer's message, Show predicted intents lists the intents your AI agent predicted. On the answer, Inspect message opens it in the inspector. In a dialogue test, Locate in dialogue builder jumps to the node that sent it. All test tools are in Testing tools in the preview.

8. Click Copy link to conversation log and send the link to the colleague who fixes the article. After the fix, add the customer's question to a test set, with the correct fact under Answer contains, as in lesson 4.

Why it matters

• The preview shows what your customer saw; the log shows what your AI agent did. Together they point to the article, dialogue or procedure to change, so you fix the cause instead of guessing.
• An answer with a placeholder such as {{first_name}} gets the check Unresolved parameter(s): customers would see that text literally.
• Quick replies mixed with link or postback buttons in one message make AI agents answer with the Technical error reply. The inspector flags that too.