Learn Before
Keep Validation and Test Label Fixes Aligned
Apply the same label-correction rule to both the validation set and the test set. If the two sets are cleaned differently, the evaluation conditions no longer match, and a model can be chosen using one labeling standard but judged using another.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
Adding a Mislabeled Column to Error Review
When to Repair Noisy Development Labels
In dev-set error analysis, what does it mean for an example to be mislabeled?
True or False: Mislabeled examples in a dev set are mistakes made by the model.
In a labeled example written as (x, _____), the second component is the target label.
Match each symbol in a mislabeled training example to its meaning.
Arrange the events that lead to finding a mislabeled validation example in a text classifier project.
What makes a dev-set example mislabeled?
Find the data problem in a traffic-sign classifier's dev set.
Where does a mislabeled dev example get its error?
Which option shows a mislabeled example in a wildlife image dataset used to classify birds?
True or False: In a mislabeled training example (x, y), the value y is wrong.
Learn After
Why should the same label-cleaning procedure be used for both the dev set and the test set?
Validation and Test Labels Must Be Handled Consistently
Any label-fixing rule you use for the development set should also be applied to the _____ labels.
Match each relabeling situation to its effect on dev and test evaluation.
Order the steps for correcting label mistakes while keeping evaluation sets aligned.
What is the main problem if the dev set and test set are labeled using different rules?
It is fine to use one procedure to clean labels for the validation set and a different procedure for the test set if both look accurate overall.
Aligning validation and test label fixes helps prevent the team from optimizing for one score and then being judged by a _____ scoring rule.
Match each label-cleaning concept to its definition.
Order the steps that create misleading evaluation when only validation labels are corrected.
Explain what happens if a team cleans labels only in the validation set and leaves the test set unchanged.
Explain the evaluation mismatch when only one dataset receives label cleanup.
Why keep label corrections aligned across development and test data?