HomeLearnCoursesHackathonsAccount
Feature Engineering for ML
Feature Selection and the Risk of Leakage · 1/2

More features isn't automatically better

It's tempting to throw every available column into a model and let it figure out what matters. In practice, too many irrelevant or redundant features can hurt performance rather than help it. Irrelevant features add noise the model has to sort through, and with limited training data, that noise can get mistaken for signal, a pattern known as overfitting. Redundant features, two columns that essentially measure the same thing, don't add new information but can still destabilize certain algorithms.

Feature selection is the deliberate process of narrowing down to the inputs that actually carry predictive signal, whether through domain knowledge, statistical tests, or techniques that rank features by how much they contribute to a trained model's predictions. The goal isn't necessarily the fewest features possible, it's the right features, ones that generalize instead of ones that just correlate with the training set by coincidence.