Case study
Evals for AI-generated output, and testing the tests
114 checks hold the bar on generated pages. Then an audit asked which of them would pass if there were nothing to check.
- checks run on every build
- 114checks run on every build
- checks that passed on an empty tree
- 13 → 2checks that passed on an empty tree
- false positives caught before acting
- 955 of 957false positives caught before acting
Figures measured 2026-10-12.
The problem
Generated output fails in ways a unit test never sees: a stuffed title, an invented statistic, a price that disagrees with the price list, a link to a page that does not exist. Each one is cheap to make and expensive to find by reading.
A green build is only worth something if every check in it can fail. A check that passes when its subject is missing is decoration.
What was built
Each defect found in review became a check over the built HTML or the source registries: metadata length, price consistency against one source of truth, broken internal links, forbidden names, accessibility of form fields, and around a hundred more.
A second-order audit runs every check against an empty tree. 13 of the first 47 passed with nothing to check. Each gained a floor (a minimum count of things it must inspect) and the number fell to 2, both documented as correct.
Measurement itself was treated as fallible. One probe reported 957 overlong list items; opening the raw HTML showed 955 were a regex swallowing nested lists. Another check carried a literal backspace byte in its pattern, so it had passed on every run without ever matching.
What changed
Defect classes stay fixed after they are found, because the build fails if one comes back.
The team rule that came out of it: open the raw output before acting on any measured finding. It has stopped at least three invented defects from reaching a pull request.
Inside your team
This is the discipline LLM evals need in production: graded checks on real output, run on every change, plus a test that the evals themselves can fail.