Creating a scorecard

Building a scorecard from real conversations rather than a blank form or a template. A critical category can fail a whole review on its own.

Building a scorecard from real conversations rather than a blank form or a template. A critical category can fail a whole review on its own.

Build it

1. Read ten recent conversations: five good, five poor, and write down what made the difference.
2. Turn those into four to six rating categories.
3. Create the scorecard: give it a Name and select the workspaces that will use it. There are two selections, not one: the workspaces that review with it by hand, and the workspaces that autoscore with it. A scorecard can serve several workspaces either way.
4. Add the categories and set the weights. A weight is a whole number from 0 to 100, and the score is each category's score times its weight, divided by the sum of the weights. Hovering over a weight shows what it comes to as a percentage of the total.
5. Test it by scoring five more conversations, adjust, and publish it.

Two things shape how you design the categories. Critical category makes one category able to fail the whole review: below fifty per cent there, and every category scores zero. Use it for the thing that genuinely cannot be got wrong, and for nothing else.

And a category does not have to be on every review. Only show on the scorecard under certain conditions makes one appear only when the conversation matches, which is how you ask about product knowledge without asking about it on a password reset.

Some categories score themselves. Spelling and grammar, comprehension, solution offered, tone of voice and closing are system categories that Zendesk QA rates automatically when they are active, so those you do not have to write from scratch.

Start from conversations, not from a template

A scorecard written from a generic template measures generic things. Reading your own conversations tells you what separates good from poor here, and it is usually not what you expected.

Test it before publishing

Score five conversations against your draft. Categories that seemed clear turn out to be ambiguous, and you will find one you forgot.

Twenty minutes, and it saves rewriting after the team has been scored on it.

Have two people score the same conversation

If they disagree, the category is not clear enough. That test catches most of the problems a scorecard has, and it catches them before anybody is affected.

Involve the agents

Show them the draft and ask what is missing or unfair. A scorecard the team helped shape is accepted; one that arrives finished is argued with, whatever is on it.

Write the guidance into the category

What counts as a pass, what counts as a fail. That text is what keeps two reviewers aligned six months from now, and it is what an agent reads when they disagree.

Publish it once

Changing a scorecard resets comparability. Get it close enough, run a quarter, then revise deliberately.

See also

• Creating manual rating categories
• What makes a scorecard that people accept
• Viewing and managing scorecards