University of Michigan - Ann Arbor

Google

The left plot shows fitting a straight line to the data. This is not a very good fit. So, it is underfitting the data and we call it high bias. On the opposite end, if you fit an incredibly complex classifier, maybe you can fit the data perfectly, but that doesn't look like a great fit either. So there's a classifier of high variance and this is overfitting the data. And there might be some classifier in between, with a medium level of complexity, that maybe fits it correctly.

Overfitting/Underfitting vs. Bias/Variance in Supervised Machine Learning

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

A supervised learning system is expected to perform well only when a chain of four assumptions holds, each verified on a different dataset:
1. **Low bias**: the model fits the training set well on the cost function (e.g., comparable to human-level performance).
2. **Low variance**: the model fits the dev (validation) set well on the cost function.
3. The model fits the test set well on the cost function.
4. The model performs well in the real world (e.g., happy users).

Because the checks are sequential, a failure at a later step points to the stage that needs attention (e.g., high dev-set error indicates a variance problem).

Chain of Assumptions in Supervised Statistical Learning

Fortmann-Roe, S. (n.d.). (2012). Understanding the Bias-Variance Tradeoff. Retrieved from
[scott.fortmann-roe.com/docs/BiasVariance.html](https://scott.fortmann-roe.com/docs/BiasVariance.html) 

Understanding the Bias-Variance Tradeoff

Training and test error behave diﬀerently. At the left end of the graph, training error and test error are both high. This is the underﬁtting regime. As we increase capacity, training error decreases, but the gap between training and generalization error increases. Eventually, the size of this gap outweighs the decrease in training error, and we enter the overﬁtting regime, where capacity is too large, above the optimal capacity.

Learn Before

Related