University of Michigan - Ann Arbor

Golden rules are human-specified design constraints that govern failure-driven harness profile evolution, preventing test-set contamination and direct answer encoding. These constraints enforce three core requirements:

1. **Minimal Reusable Control**: Prioritize minimal, reusable procedural interventions rather than maximal, rigid workflows.
2. **Semantic Routing**: Route execution, state transitions, and verification checks based strictly on visible task semantics and observable workspace evidence, never on task identifiers, names, hashes, hidden tests, or verifier code.
3. **Evaluation Integrity**: Strictly separate development feedback from frozen evaluation, ensuring that candidate runbooks are frozen prior to held-out benchmark evaluation.

Golden rules govern the meta-level evolution of the control profile and are distinct from the operational checks evaluated during an agent execution.

Golden Rules for Harness Profile Development

Under the golden rules for harness profile development, which requirement prohibits conditioning state transitions or verification checks on task names, task hashes, or hidden test code?

Learn Before

Related