Creating a scorecard

Building a scorecard, starting from real conversations rather than from a blank form or a template.

Building a scorecard, starting from real conversations rather than from a blank form or a template.

Build it

1. Read ten recent conversations: five good, five poor.
2. Write down what made the difference.
3. Turn those into four to six categories.
4. Create the scorecard and set weights.
5. Test it by scoring five more conversations.
6. Adjust, then publish it to the workspace.

Start from conversations, not from a template

A scorecard written from a generic template measures generic things. Reading your own conversations tells you what actually separates good from poor here, and it is usually not what you expected.

Test it before publishing

Score five conversations against your draft. Categories that seemed clear turn out to be ambiguous, and you will find one you forgot.

Twenty minutes, and it saves rewriting after the team has been scored on it.

Have two people score the same conversation

If they disagree, the category is not clear enough. That test catches most of the problems a scorecard has, and it catches them before anybody is affected.

Involve the agents

Show them the draft and ask what is missing or unfair. A scorecard the team helped shape is accepted; one that arrives finished is argued with, whatever is on it.

Write the guidance into the category

What counts as a pass, what counts as a fail. That text is what keeps two reviewers aligned six months from now, and it is what an agent reads when they disagree.

Publish it once

Changing a scorecard resets comparability. Get it close enough, run a quarter, then revise deliberately.

See also

• Creating manual rating categories
• What makes a scorecard that people accept
• Viewing and managing scorecards