Classical ML

Expected test error splits into $\text{Bias}^2$, variance, and irreducible noise — knowing which term dominates tells you whether to add capacity or add data

Distance-based, gradient-trained, and penalized models all silently assume features share a scale — know which models care, which transforms to use, and why the scaler must be fit on the training set only

Penalizing $\|\beta\|_1$ zeroes coefficients out, penalizing $\|\beta\|_2^2$ shrinks them smoothly — the geometry of corners explains why, and the choice changes how correlated features are handled

A model is only as good as its error on data it never saw — how to detect memorization with learning curves, protect the test set, and spot leakage before production does

OLS in interview form: the assumptions and which ones matter for prediction vs inference, reading coefficients without lying, catching multicollinearity with VIF, and knowing when $R^2$ is the wrong scoreboard

A linear model pushed through a sigmoid: coefficients are log-odds, $e^\beta$ is an odds ratio, and cross-entropy — not MSE — is the loss, for reasons every interviewer asks about

The two simplest strong baselines: k-nearest neighbors memorizes the data and votes at query time, while Naive Bayes multiplies per-feature evidence under a deliberately false independence assumption.

Maximum-margin classification from hinge loss to the dual formulation, and why an algorithm that only touches inner products gets nonlinear boundaries almost for free via kernels.

When positives are rare, default thresholds and ROC curves flatter the model — threshold tuning, class weights, resampling, and PR-AUC are the tools, and cost-sensitive framing is the principle that unifies them

Precision, recall, F1, and ROC-AUC from one confusion matrix — what each number actually measures, why accuracy misleads, and how to read AUC as a ranking probability

k-fold, stratified, leave-one-out, time-series splits, and nested CV — which protocol estimates generalization honestly, and the leakage traps that make CV numbers lie

Greedy splits scored by Gini or entropy build an interpretable but high-variance learner; bagging plus feature subsampling turns a crowd of them into a forest whose variance shrinks like $\rho\sigma^2 + (1-\rho)\sigma^2/B$

Fit a small tree to the current errors, add a shrunken copy, repeat — gradient descent in function space. The bias-killer to bagging's variance-killer, and the reason a learning rate exists in a tree model

k-means mechanics step by step, how to pick k with elbow and silhouette, and the decision map between k-means, GMM, and hierarchical when the blobs are not round