An unknown value became 0%.
State had once converted an unresolved automation target into a confident 0%. The output fit the expected structure, but it invented certainty the project did not have. That made the eval question more useful than simple accuracy: what kinds of mistakes matter enough to block release?
I separated correctness into four product behaviors.
A fluent answer does not count if the evidence does not establish it.
Missing consequential new information can be as harmful as adding something false.
Unresolved or conflicting evidence should not quietly become a settled fact.
A system that sends everything to Review can look safe while making the product impractical to use.
Not every failure deserved the same tolerance.
For this exercise, unsupported consequential claims and meaningful loss of uncertainty were release-blocking. A strong average score would not cancel out one severe false-certainty failure.
I did not want to “fix” uncertainty by escalating everything. The exercise used a product-specific 20% ceiling for clearly unnecessary Reviews to keep the system useful as well as cautious.
I established the expected behavior before looking at simulated model outputs. AI helped generate cases, but it did not decide what counted as acceptable behavior.
The baseline and post-fix result numbers in this learning exercise were simulated for decision practice. They are not measured State production performance.
A failed eval tells me something is wrong, but not necessarily where.
Working backward through that chain exposed two gaps. I needed tracing to connect a bad output to the first observable failure instead of assuming the prompt was wrong. I also needed a way to audit silent misses, because evidence incorrectly classified as “no Review needed” may produce no visible failure at all.
Retesting one broken sentence is not enough.
Confirm the specific regression is gone before widening the test.
A narrow prompt patch should not pass only when the wording looks like the example that inspired it.
Use unseen examples and repeated runs for the behaviors where model variability matters.
AI evals are product decisions expressed as tests.
The useful outcome was not one accuracy number. The exercise forced me to decide what State could tolerate, what should block release, and how much human-review burden was acceptable. It also made me name the evidence that would cause me to change those decisions.
What I did vs. what AI did
I chose the behavior to evaluate and defined what acceptable behavior meant. I set the severity and review-burden thresholds, then decided what the results meant for the product.
I used AI to draft cases, generate simulated outputs, suggest nearby examples, and organize the follow-up work.