Single-Score Model Evaluation

An Early Dev/Test Setup Speeds Model Iteration

Revise the evaluation setup when it stops matching the goal

Error Analysis

Reviewing Both Missed and Correct Dev-Set Examples

After a development set and a test set have been chosen, what are they mainly used for?

True or False: A development set score can provide a quick check on whether a new idea is moving a model in the right direction.

Dev and test sets give a quick check on how well a team’s _____ is performing.

Match each dev/test-set action with its main effect.

Arrange the basic workflow for using development and test sets to improve a model.

Explain why a dev set, test set, and metric help a machine learning team work efficiently.

Diagnosing a project with no evaluation set

In one to three sentences, explain how a validation set can guide which model ideas deserve more work.

How does dev set performance help a team choose which ideas to keep improving?

What do teams typically do after they have set up dev and test sets?

True or False: With a fixed development set and a single evaluation metric, a team can quickly tell whether a new idea is helping by a little or by a lot.

A development set and a chosen evaluation metric help a team compare ideas quickly and see whether each change is making progress.