🚧 Feature is in beta and available on request
Beta means the feature is still being refined based on customer feedback, so small changes may occur as we improve it.
To get access, contact support@voyc.ai or your Voyc Account Manager.
What an LLM Scorecard Is and Why It Matters
LLM scorecards help you configure scorecards faster and get more accurate results from day one. They reduce the guesswork in setup, cut down on manual tweaking and give you scoring that keeps up with how people really speak. This means smoother onboarding, quicker time to value and stronger confidence in the insights you use for coaching and QA.
An LLM scorecard adds a new intelligence layer on top of Voyc’s existing routing and scoring engine. It uses our custom large language model (AI) together with our Gen 4 scripting to understand context, follow complex logic and match statements to the right conditions more reliably.
Instead of focusing mainly on keywords, the LLM pays attention to meaning. It can recognise when agents and customers express the same idea in different ways, which makes configuration easier and matching more accurate without changing how the core scoring engine works.
LLM scorecards also power ALEX Live. You need at least one LLM scorecard set up to unlock real time insights, since the scorecard provides the intelligence ALEX reads during each conversation.
In short, LLM scorecards speed up setup, improve accuracy and unlock features like ALEX Live, all while building on the scoring engine you already trust.
📝 Note: LLM scorecards power ALEX Live. You need at least one LLM scorecard configured before you can use ALEX Live in your workspace
How to Create an LLM Scorecard
LLM scorecards work just like your normal scorecards, with added LLM and gen 4 scripting intelligence layered in. If you already know how to build a standard scorecard, this will feel very familiar, only smarter and quicker. For the basics of groups, nodes and weighting, start with How to Create a Scorecard and then come back here for the LLM specific setup.
Access the Scorecard Builder
Go to your Channel.
Click Configurations.
Open the Scorecard tab.
Select + Add Real-Time Call Guide.
In the pop up, enter a Name and optional Description, then click Create.
You are now in the LLM scorecard builder.
Create a Group and Statement node
On the Start block, click the + then +Group to add a Group.
Click on the canvas to place the group.
Give the group a clear Name so anyone can see what it covers.
Set the Group points, then click Save.
Next to the group name, click the + to add a Statement Node.
Give the Node a clear Name so anyone can see what it covers and click Save.
The LLM statement node will open for editing.
From here you are working inside a single LLM statement node.
How to Configure the LLM Statement Node
1. Choose the statement type
Set the Statement Node Type to Compliance, Sales or both, using the toggle.
Pick the type that matches what this statement is measuring, this helps the LLM decide what it should be listening for.
2. Add assessment criteria
In the Assessment criteria box, describe what the LLM should check for.
For example:
“Check and confirm all details that form the basis of the policy, ensuring no inaccuracies remain before proceeding.”
“Ascertain if the customer has got a quote from another insurer already. Checks for competitor awareness and buying stage.”
Use clear, natural language. This text is the AI prompt for that node, you do not need to detail everything.
Click Save.
The LLM uses this to decide whether the statement has been met.
3. Add Scripting
The scripting helps the LLM to make decisions on your routing and scoring. The scripting is also what will be shown to the Agent in ALEX Live, you can choose for the script to be verbatim if the Agent needs to say it word-for-word.
In the Scripting block, insert the part of your script that applies to this statement.
Click Save.
Turn on the Verbatim toggle if the agent must read it word-for-word.
4. Fine Tune Statement Options
As with a normal scorecard, you can now:
Add Keywords if you want to reinforce specific phrases.
You can Ask Joyc for suggestions.
Keywords are optional for LLM statements. The LLM can rely on context alone.
Set the Node weight to control how important this statement is.
Toggle Auto fail if this is a critical compliance requirement.
Click outside the node to close it.
5. Build the Rest of the Scorecard
Add additional statement nodes to the group as needed and link them in the flow.
Create more groups for different parts of your process, just as you would in a normal scorecard.
Functionally, the LLM scorecard behaves like your normal scorecard. The difference is that each LLM statement has richer criteria and scripting, which the model uses to score conversations more accurately and power ALEX Live.
The Gist
LLM scorecards help you configure scorecards faster and get more accurate results from day one by adding Voyc’s custom LLM (AI) and Gen 4 scripting on top of the scoring engine you already trust. They understand context, follow complex logic and match statements more reliably, so your QA feels sharper and less manual. They also power ALEX Live, unlocking real time coaching insights that keep up with how people actually speak, not just which keywords they use.