LLM Output Evaluation Scorecard: From “Looks Good” to Release Evidence

An LLM evaluation should answer a release question: is this exact system configuration good enough for this defined use, and what evidence supports that decision? “The new model sounds better” is not evidence. The system includes the model version, instructions, retrieval, tools, temperature, safety controls, output parser, and human review path.

This scorecard uses three layers: hard failures that block release, task-specific quality dimensions, and operational measurements. It can be run manually in a spreadsheet or translated into an evaluation service after the rubric is stable.

Write the decision and failure budget first

Define the task, users, environment, downstream action, and harm from an error. Then write the release decision in one sentence: “Approve configuration B for internal first-draft support replies, with human approval, if it has no privacy or fabricated-policy failures and meets the per-stratum thresholds.” This is narrower and more auditable than “approve model B.”

Anthropic's evaluation guidance recommends specific, measurable, achievable, and relevant criteria and notes that most LLM applications need multiple dimensions. Translate that into a failure budget. A casual rewriting tool may tolerate occasional style misses; a system that quotes policy, changes an account, or gives health guidance needs different gates and human control.

Build a stratified test set

Start with 50 to 100 cases for an internal pilot, then expand from production failures. Divide them into strata so an average cannot hide a dangerous subgroup:

• Typical requests that represent normal traffic.
• Rare but valid requests with long, messy, multilingual, or ambiguous input.
• Adversarial requests: prompt injection, policy evasion, data extraction, and tool misuse.
• Unanswerable cases where the correct behavior is to ask, abstain, or escalate.
• Critical entities: names, dates, money, dosage, identifiers, citations, and negation.
• Known regressions copied from real defects after removing personal data.

Each row needs an input, allowed context, expected behavior, prohibited behavior, reference facts, severity, stratum, and reviewer notes. Avoid one perfect reference paragraph when multiple answers are valid; specify required facts and forbidden claims instead.

Separate hard failures from scored dimensions

A weighted average should never compensate for a privacy leak or fabricated citation. Mark these as hard failures when relevant: disclosing restricted data; executing an unauthorized action; inventing a source, quote, person, policy, or transaction; violating a must-follow instruction; producing disallowed safety content; or presenting an unsupported high-stakes answer without escalation.

Score quality only after the hard-failure check.

Dimension	0	2	4	Suggested weight
Factual support	Contradicts evidence	Mixed or weakly supported	Every material claim supported	25%
Task completion	Misses objective	Partial completion	Completes all required steps	20%
Instruction following	Breaks hard constraint	Minor format miss	Meets all constraints	15%
Uncertainty	Invents confidence	Vague caveat	Correctly abstains or scopes	15%
Evidence use	Fake or irrelevant citation	Incomplete mapping	Claims map to supplied evidence	15%
Clarity	Misleading or unusable	Needs significant edit	Direct and usable	10%

Rewrite anchors for the domain. A support assistant's task-completion rubric might require correct policy, next step, and escalation route. A coding agent needs tests, scope control, and no unrelated file changes.

Calibrate human reviewers

Have at least two qualified reviewers score the same initial sample independently. Compare disagreements by dimension, not only total score. Refine ambiguous anchors, add examples, and repeat until reviewers interpret the rubric consistently enough for the decision. Preserve both raw scores and the adjudicated result.

Blind reviewers to model identity and randomize output order where practical. Do not let the person who built a prompt be the only evaluator. Domain experts must judge domain correctness; generic crowd preference is not a substitute for legal, medical, security, or financial expertise.

Use model-based graders only after calibration

OpenAI's Evals API supports datasets and several grader types, including string checks and model graders. Google documents judge-model evaluation against human ratings and provides balanced accuracy, F1, and confusion-matrix methods for checking whether the judge agrees with people. Those capabilities are useful for scale, but the judge is another model with its own bias, prompt sensitivity, and failure modes.

First build a human-rated calibration set. Test the automated grader separately on positive, negative, ambiguous, and adversarial cases. Monitor false passes, especially for fabricated evidence and subtle policy violations. A high agreement percentage can hide poor performance on the rare cases that matter most, so report results by stratum and failure type.

Measure operations alongside answer quality

For every case, log configuration ID, model version, prompt version, retrieved source IDs, tool calls, tokens, latency, retries, parser success, safety intervention, human edit time, and cost. Use percentiles for latency rather than one average. Track abstention and escalation rates because a system can inflate apparent accuracy by refusing everything.

Measure end-to-end behavior. A correct draft that calls the wrong tool, exposes data in logs, or fails the output parser is a failed system result.

Protect the test set from leakage and drift

Separate a development set used for prompt iteration from a sealed holdout used for release. If builders repeatedly inspect holdout failures and tune against them, the holdout becomes another development set. Rotate a portion of cases, retain a stable regression core, and add production-like cases from a documented sampling process.

Review labels as policies and source documents change. A model should not be penalized for following a new rule against an old reference answer, nor rewarded for quoting a policy that is no longer active. Version the dataset, references, rubric, and adjudication decisions together. When comparing releases, disclose any rows or labels that changed.

Set release and rollback thresholds

Require zero hard failures in the critical set, minimum scores for each critical dimension, and per-stratum thresholds. Compare candidate and baseline on the same rows. Review regressions individually; do not ship a small average improvement if it creates a new severe failure.

Pin the evaluated model version where the provider supports it, retain the configuration, and make rollback possible. Re-run the full regression set on prompt, model, retrieval, tool, policy, parser, or data changes. Add every escaped production defect as a sanitized test case.

Original worksheet

Download the LLM output evaluation scorecard (CSV). It contains hard-failure flags, six anchored dimensions, strata, reviewer fields, baseline and candidate IDs, operational metrics, adjudication, release decision, and regression owner.

Source map

• Anthropic's success-criteria guide supports the specific, measurable, task-relevant criteria used here.
• OpenAI's Evals reference supports the dataset and testing-criteria workflow; the Graders reference supports the cited grader categories.
• Google's judge-model guide supports calibration against human ratings and the listed comparison metrics.
• NIST's AI RMF Playbook supports documenting actions across govern, map, measure, and manage.

Limitations

This scorecard was source-reviewed on July 17, 2026. The suggested dimensions, weights, sample size, and thresholds are starting points, not standards. A test set cannot prove correctness outside its coverage. Human raters and judge models can both be biased, and vendor evaluation APIs can change. High-stakes deployment requires domain-specific validation, independent review, monitoring, incident response, and applicable regulatory assessment.