Harness scaling investigates whether external execution controls can convert a model's latent capabilities into reliable work through three progressively stronger empirical evaluations:

1. **Fixed-Model Improvement (Within-Model Lift)**: Assesses whether a refined execution harness can improve the end-to-end task completion rate of a fixed model without altering its underlying weights.
2. **Cross-Generation Transfer (Frozen Transfer)**: Assesses whether an execution control profile developed on an earlier model generation can transfer directly to a newer, more capable model without retuning or modifying control logic.
3. **Beyond-Benchmark Generalization (Held-Out Task Generalization)**: Assesses whether the resulting control principles and runtime mechanisms can improve performance on completely different, held-out task distributions beyond the benchmark on which they were originally developed.

Three Empirical Tests of Harness Scaling

The control-layer design space maps agent architectures along two primary operational axes: runtime enforceability (the degree to which the host runtime maintains authoritative state, constrains valid transitions, and blocks incomplete handoffs) and agent autonomy (the latitude retained by the primary agent to select tools, revise procedures, and operate across a broad action space). A third dimension, control-layer accessibility, distinguishes whether the control artifact is developer-owned, agent-editable, or jointly visible and operable by both agent and human supervisor.

Control-Layer Design Space for Long-Horizon Agents

Harness scaling is the systematic enhancement of the execution and control layer surrounding an autonomous agent without changing its underlying model weights. Rather than replacing model scaling, harness scaling investigates how much of an agent's latent capability can be converted into completed, reliable work by improving runtime mechanisms that maintain durable state, constrain valid execution, verify progress before transitions, and recover from errors.

Harness Scaling

Control-signal dilution is an operational execution failure mode and design hypothesis in long-horizon autonomous agents where the guiding influence of a compact plan and its completion criteria weakens as execution tokens accumulate. As an agent generates a long append-only trace of commands, observations, and intermediate repairs, the high-level plan tokens receive diminishing attention relative to the surrounding trace. This dilution causes the agent to deviate from its plan, repeat unproductive steps, or terminate prematurely before satisfying task obligations. External runtime harnesses mitigate this failure by externalizing procedural state, refreshing phase-relevant control contexts upon state entry, and verifying progress before transitions.

Control-Signal Dilution

Mutable-state ambiguity is an operational execution failure mode and design hypothesis where an autonomous agent struggles to ascertain completed goals, pending dependencies, prior failed attempts, and valid next actions because it must reconstruct them from an append-only interaction history rather than reading an authoritative current state. In complex workflows involving loops, branching, and repeated repairs, interleaved updates obscure the live system state. External runtime harnesses resolve this ambiguity by tracking durable execution state outside model context, establishing explicit context-and-contract boundaries, and enforcing checked transitions.

Mutable-State Ambiguity

A procedural-memory gap is a longitudinal failure mode occurring across independent task executions where an agent correctly diagnoses an operational failure in an earlier run but neither retains nor invokes the resulting lesson when the same risk recurs. It contrasts with an epistemic gap (where necessary knowledge or methods are unavailable at decision time) and a procedural-compliance gap (where the correct procedure is active but remains incomplete or not followed).

Procedural-Memory Gap in Autonomous Agents

University of Michigan - Ann Arbor

Ch.1 Foundations and Operational Challenges - Long-Horizon Agent Reliability: Stateful Scaffolding and Runtime Verification @ University of Michigan - Ann Arbor

Operational Gaps in Long-Horizon Agent Execution - Long-Horizon Agent Reliability: Stateful Scaffolding and Runtime Verification @ University of Michigan - Ann Arbor

Harness Scaling Foundations and Control Design Space - Long-Horizon Agent Reliability: Stateful Scaffolding and Runtime Verification @ University of Michigan - Ann Arbor

Learn Before

Related

Learn After