What rating categories are

One question on a scorecard. Whether it can be answered from the transcript decides whether it works.

One question on a scorecard. Whether it can be answered from the transcript decides whether it works.

What a category is

A single question about the conversation, with a scale and guidance on what each score means. The scale is binary, a 3-point or a 4-point one, and it is set on the scorecard rather than on the category itself. Several categories together make a scorecard.

The test: can you point at the evidence

"Did the agent confirm the issue before answering?" can be checked against the text. "Was the agent professional?" cannot, and two reviewers will score it differently every time.

If a category cannot be evidenced, it produces disagreement rather than measurement.

Categories worth having

• Was the answer correct? The one that matters most and is often left off.
• Was the issue understood before answering?
• Was the reply clear and complete?
• Was the process followed where one exists?
• Was the tone right for the situation?

Four kinds, not two

• Manual. A reviewer reads and decides. Correctness and understanding stay here, because they need somebody who knows the subject.
• AutoQA system categories. Eight of them, scored for you: greeting, closing, empathy, tone, comprehension, solution offered, spelling and grammar, and readability.
• AI prompt-based. You describe in plain language what should be judged and it is scored automatically. Both automatic scoring and LLM-based AutoQA have to be switched on. Ten active ones per account, and that ten is shared with AI prompt-based spotlight insights, so insights you set up elsewhere use up the same allowance. At the limit there is no queue: you mark one inactive or delete it before you can add another.
• Exact text-match. You list the keywords or phrases, choose whether to search the agent's messages, the customer's or any message, and set what a match scores and what a miss scores: thumbs up, thumbs down or not applicable. Twenty active ones per account, which is a separate allowance from the ten above. The matching is literal, so add the variations your team writes rather than the one you would write.

The automatic ones exist to free reviewer attention for the judgement calls, which is the whole reason to use them.

Write the guidance

What a pass looks like, what a fail looks like, with an example of each. That text is what keeps reviewers aligned and what an agent reads when they disagree.

Critical categories

One or two, where a failure invalidates the interaction: wrong information, a policy breach. Marking many categories critical means none of them carry weight.

See also

• Creating manual rating categories
• What AutoQA does
• What a scorecard is