Classical ML

Fundamentals

The tradeoffs underneath every model — generalization, regularization, and getting the inputs right

Review this topic

Bias-variance tradeoff
core
Expected test error splits into $\text{Bias}^2$ , variance, and irreducible noise — knowing which term dominates tells you whether to add capacity or add data
12 minfundamentalsgeneralization
Feature scaling and preprocessing
intro
Distance-based, gradient-trained, and penalized models all silently assume features share a scale — know which models care, which transforms to use, and why the scaler must be fit on the training set only
8 minfundamentalspreprocessing
L1 and L2 regularization
core
Penalizing $\|\beta\|_1$ zeroes coefficients out, penalizing $\|\beta\|_2^2$ shrinks them smoothly — the geometry of corners explains why, and the choice changes how correlated features are handled
11 minfundamentalsregularization
Overfitting and generalization
core
A model is only as good as its error on data it never saw — how to detect memorization with learning curves, protect the test set, and spot leakage before production does
9 minfundamentalsgeneralization