HomeLearnCoursesHackathonsAccount
Supervised Learning in Practice
Overfitting and Underfitting · 1/2

Two ways a model can fail, in opposite directions

Overfitting happens when a model learns the training data too well, including its noise and one-off quirks, rather than the real underlying pattern. The telltale sign is a big gap between training and test performance: the model scores great on data it has seen and noticeably worse on new data, because it essentially memorized specifics that don't generalize. A decision tree allowed to grow very deep, splitting on smaller and smaller subsets of data, is a classic example, it can carve out a rule for nearly every individual training row.

Underfitting is the opposite failure: the model is too simple to capture the real pattern in the first place, and performs poorly on both training and test data. Trying to fit a straight line to a relationship that's genuinely curved is a common example, linear regression will underperform everywhere because the model's basic shape can't represent what's actually going on in the data, no matter how much training it gets.