Plan requirement
| Subscription | Any plan |
| Also required | Quality Assurance or Workforce Engagement Management |
| Access | Admin |
Writing a category reviewers can apply consistently. The guidance text is most of the work.
Create one
- Open the rating categories in the workspace.
- Add a category and write the question.
- Choose the scale.
- Write what each score means, with an example.
- Have two reviewers score the same conversation with it.
Ask one thing
"Was the reply accurate and complete?" is two questions. A reviewer facing a reply that was accurate and incomplete has no honest score to give.
Split it, or accept that the category will be applied inconsistently.
Write it as a question about the conversation
Not about the agent. "Did the reply address the customer's actual question" is checkable. "Does this agent understand our products" is a judgement about a person from one exchange.
The guidance is what makes it work
Two or three lines per score point, with an example from a real conversation. Without it, each reviewer builds their own private standard and the scores stop meaning one thing.
This is the part people skip and the part that determines whether the category survives.
Keep the scale simple
Three points is usually enough: below, meets, exceeds. A five or ten point scale gives an illusion of precision and produces clustering in the middle.
Test with two reviewers
Same conversation, scored independently. If they disagree, rewrite the guidance rather than assuming they will converge with practice.
Allow not applicable
Some categories do not apply to some conversations. Without that option, reviewers score something arbitrary, and the average is polluted.
Comments
0 comments
Article is closed for comments.