How to build an AI scorecard

They say practice makes perfect, and well-structured feedback makes perfect practice.

This guide covers how to build an AI scorecard that pushes reps to improve faster, and how to attach reference files so the AI scores against your documents instead of guessing. If you follow MEDDPICC, GAP, Challenger or another methodology, the scorecard is where that framework goes into PitchMonster.

Where to find it

The scorecard is step 4 of 5 in the role-play editor: Set the goal and scoring.

To reach it on a role-play you already built, find the card on your dashboard or in the Role-plays tab, click the three dots in the top right corner of the card, and choose Edit role-play:




The editor opens, and the Scorecard step is waiting on the left.

If you're building a new role-play, you land on this step as part of the flow.

Two buttons sit at the top of that screen:

Choose Template pulls in a scorecard you've already built. Save as template stores the one you're working on so your team can reuse it on the next role-play. Building the templates you use most is worth doing early, because it turns a twenty-minute setup into two clicks.

Anatomy of a PitchMonster AI scorecard

And here is how the AI uses them:

Each metric is scored 1 to 5, so the total stays balanced whether the metric holds two points or six. Add up all metric scores and you get an overall score out of the best possible result. The AI ticks off any point it detects in the conversation. When it misses a point, it adds a coaching comment so the rep knows what to fix.

1. Goal

Describe what the rep should achieve in this role-play. One sentence. The field caps at 150 characters, so cut it to the outcome: booked a next step, uncovered budget, got the security team on the call.

The goal is required. You can't publish the role-play without it.

2. Metrics

Add the four to six things a rep has to nail before you'd call the conversation productive:

• Intro
• Discovery
• Pitch / Value prop
• Objection handling
• Next steps

Click Add Metric for each one. Drag the handle on the left to reorder them into the sequence your call actually follows. The number on the right of each metric row shows how many points sit inside it, and the chevron collapses the metric once you're done with it.

The Tutorial button in the Metrics header opens this guide, so your team can pull it up mid-build without leaving the editor.

3. Points

Break each metric into 2 to 6 points. For discovery, for example:

• Uncovered current solution
• Surfaced two pain points
• Quantified business impact

Click the + inside a metric to add a point.

4. Context

Context is optional, and it matters most when phrasing or depth decide whether a point was really hit. If the point can't be judged from a single phrase, add context. Click the speech bubble icon next to the point to open the context box.

Your input feeds the AI-generated comments your reps read afterwards.

You don't need context for "Ask customer name". The point speaks for itself.

You do need context for "Ask customer about current setup". A rep saying "Hey, how do you do it now?" might not clear the bar your sales process sets. Spell out what does:

Rep must:

• Ask, "Walk me through your current CS onboarding workflow."
• Follow up: "On average, how many reps onboard per month?"
• Calculate total hours and cost (rep count x 10 hrs x manager hourly rate).

The AI will only score the point when the rep walks that flow and captures the numbers, the way your best rep would.

Things worth putting in the context box:

• Example questions or phrases
• Pieces of script
• Granular points or the flow you expect
• A reference to learning material

You can set your level of expectation at the top of the box:

• Verbatim to [questions/phrases]
• Similar to [questions/phrases]
• Rep should mention [granular points]
• If the rep failed to address this point, recommend [learning material]

5. Knowledge base

The knowledge base sits next to the Goal field on the same screen. Click Edit knowledge base, upload your files, then click Done. To add files to a role-play that's already live, open it through the three dots on the card, choose Edit role-play, and go to the Scorecard step.

Once files are attached, the AI treats them as the source of truth while scoring and coaching. It checks facts, numbers, specs and product details against your documents rather than working from the prompt alone. If a rep states something that contradicts them, the AI catches the mismatch and coaches on it.

Say a rep claims "0 to 100 km/h in 5 seconds" and your product sheet says 4 seconds. The AI flags the gap in the feedback.

A few limits worth knowing:

• Up to 10 files per scorecard.
• PDF, DOCX, PPTX, XLSX, CSV and TXT are supported, and you can mix formats in the same scorecard.
• Knowledge bases aren't shared between scorecards. Each one keeps its own files, so different role-plays can point at different documents.

Common mistakes

Too many metrics. Past six, reps drown in things to remember at once. Cap it, or split the conversation into separate role-plays.

Tracking non-verbal cues. The AI understands what was said. It doesn't judge delivery well yet. "Assume the sale, speak with a confident and low voice" won't produce fair feedback.

Time-based criteria. The AI has no sense of time. Swap "Tell value prop in under 15 seconds" for "Tell value prop at the start of the call in a few sentences."

Ambiguous wording. The AI follows your instructions literally. Use explicit verbs: asks, calculates, books.

Missing context where it counts. If you sell ERP into enterprise, "Uncover a few pain points" tells the AI almost nothing. Points like that are worth the extra five minutes. Context is the single biggest lever on how well role-plays work for your organisation.

Iteration loop

1. Build v1.
2. Run five recordings with real users. Note the false positives and false negatives.
3. Tweak the wording.
4. After two or three cycles you'll have a scorecard worth saving as a template and reusing everywhere.

TL;DR

A strong AI scorecard is your playbook broken into metrics, then concrete points, with context where phrasing matters and files behind it where facts matter. Build it, iterate, save it as a template, and watch reps level up without another hour of manager talk-time.

Questions, or want a live teardown? Reach out to your PitchMonster Customer Success Manager.