Google

When a dataset contains only a very small number of labeled examples, success depends heavily on how the inputs are represented. With roughly 20 training examples, careful feature design can matter more than choosing between logistic regression and a small neural network. Either model can work better or worse depending on those features.

Tiny Training Sets Make Feature Design Crucial

With only about 20 labeled examples, which factor is likely to matter more?

A hand-crafted classifier is always superior to a neural network when the training set is very small.

With about 20 examples, _____ can matter more than algorithm choice.

Match each low-data idea with its meaning.

Question:
In a brief analytical response, explain why the decision between logistic regression and a neural network may matter less than careful feature design when you have only around 25 labeled examples.

Sample answer:
With only around 25 labeled examples, how the inputs are represented can have a larger impact on performance than the choice between logistic regression and a neural network. In this setting, thoughtful feature design may help either model much more than switching from one algorithm to the other. Neither model is guaranteed to win, so there is no automatic reason to expect the neural network to beat the simpler method. Also, the advantage of neural networks is much less reliable on tiny datasets than it is when there is abundant training data. For that reason, feature engineering deserves close attention in very small data problems.

Key points:
- Very small datasets make feature design especially important
- Logistic regression and a neural network may perform similarly
- There is no guaranteed winner based only on model type
- Neural networks are not consistently better when data is scarce

Rubric:
A strong response identifies the very small data regime, explains that feature design may matter more than the choice between logistic regression and a neural network, avoids claiming a guaranteed winner, and notes that neural-network advantages are less dependable with scarce data than with large datasets.

Why feature design can matter more than model choice when data is scarce

Case context:
A product team has roughly 20 labeled defect photos from a new inspection line. It is debating whether to use a linear classifier or a small neural network, but it has barely considered how to represent the images through hand-crafted features.

Question:
From the evidence given, what should the team treat as the more important issue, and what mistaken assumption should it avoid?

Sample answer:
The team should treat feature design as the more important issue, because with only about 20 examples, careful feature construction can matter more than choosing between a linear classifier and a neural network. The team should avoid assuming that either model family will automatically be the better choice in such a small-data setting.

Key points:
- The dataset is extremely small.
- Feature engineering may have the biggest impact on performance.
- The specific model family may matter less than the features.
- Neither approach is guaranteed to win automatically.

Rubric:
Full credit requires naming feature design as the priority, linking that priority to the roughly 20-example situation, and rejecting the idea that one model family is certain to outperform the other.

Identify the main concern for a team with only 20 labeled examples.

Question:
Answer in one to three sentences: Why might a simpler, traditional model sometimes outperform a neural network when labeled data is extremely limited, but not always?

Sample answer:
The result depends heavily on how informative the engineered features are. With only a tiny dataset, good feature design can matter more than whether the learner is a logistic regression model, a decision tree, or a neural network.

Key points:
- The outcome depends on the quality of the hand-built features.
- When the dataset is tiny, the specific algorithm family may matter less than the input representation.

Rubric:
The response should explain that feature engineering strongly affects performance in the small-data setting and that this can outweigh the choice of model.

Why can a simple model’s advantage with very little data be hard to predict?

What is the most accurate takeaway about model performance when data size changes?

When a dataset contains only about 20 labeled examples, careful feature design can matter more than choosing between a linear model and a small neural network.

Learn Before

Related