Google

Optimization verification does not depend on having a perfect reference output. If you can obtain a human-produced result that is clearly better than the current model output, it can still serve as the comparison point. That comparison can reveal whether the main problem is the optimization procedure or the scoring objective, even when the human result is not globally optimal.

A useful human output for optimization verification

Which property should a human reference trajectory have in optimization verification?

To verify an optimization result, the comparison output must be the best possible output available.

A verification comparison output must be provably optimal.

The candidate output must be _____ when compared with the current model's output.

Match each optimization verification term with its meaning.

Question:
In a concise analytical response, explain why a human demonstration path can still be useful in a comparison test even if it is not the best possible path.

Sample answer:
A human demonstration does not need to be globally optimal for the comparison test to be useful. It only needs to be better than the current system output. If the demonstration outperforms the model, the comparison can still reveal whether the main limitation is in the search or optimization method, or in the scoring rule. That makes a superior reference sufficient for deciding what to improve next.

Key points:
- The reference can be good without being optimal.
- It must outperform the current model output.
- A superior reference is enough for the test.
- The result helps decide whether to improve search or scoring.

Rubric:
A strong response explains the difference between superior and optimal, connects the human demonstration to the current system's performance, and states how the comparison guides the next improvement choice.

Explain why a better reference is enough for this comparison test.

Case context:
A package-delivery drone follows a route produced by a learning system. A human planner has drawn a route that is clearly better on the same city map, but no one can prove that it is globally optimal.

Question:
Decide whether the human route can be used as the reference y* for Optimization Verification and explain what the test can reveal.

Sample answer:
Yes. The human route can be used as y* because it is better than the current system's route, even though it has not been proven optimal. The test can help determine whether improving the optimization method or improving the scoring function is the more promising direction.

Key points:
- The human route outperforms the current algorithm output.
- Global optimality is not required.
- The human route can still serve as y*.
- The comparison helps choose between optimization and scoring improvements.

Rubric:
The response should accept the human route as a valid reference because it is better than the current output, avoid claiming that it is globally optimal, and state that the test helps decide between improving optimization and improving the scoring function.

Can a better human route serve as the reference in Optimization Verification?

Question:
Answer in one to three sentences: What must the reference output y* satisfy, and what is not necessary?

Sample answer:
y* should be a result that scores better than the model's current output under the chosen evaluation rule. It does not need to be the single best possible answer, only an answer that is better for comparison.

Key points:
- y* must beat the present model output.
- y* does not have to be globally optimal.

Rubric:
The response must include both the comparison to the current output and the fact that global optimality is unnecessary.

Learn Before

Related