PLAYER 01 · HUMAN IN COMMAND
This is not a model pretending to grade itself. Pilot one sanitized engineering change through explicit criteria, independent AI reviewers, correction loops, deterministic evidence, and final human sign-off.
The mission starts with a real problem, constraints, and acceptance criteria. The agent is not allowed to redefine success after seeing its own output.
ROLE-SPECIFIC REVIEWERS
FLIGHT MANUAL / WHY THIS LOOP EXISTS
A model is useful only when it works for the application’s intended purpose. Public benchmarks do not replace product-specific criteria and cases.
Criteria, rubrics, examples, and failure severity are defined before judgment. Otherwise “good” moves whenever the output changes.
AI reviewers are probabilistic systems. Their prompts, models, agreement with humans, false positives, and misses require evaluation too.
Automated judges scale review; deterministic tests establish facts; a human owns ambiguity, risk, and the final acceptance decision.