ML Interview Cheat Sheet

The night-before quick reference: formulas, tables and the numbers interviewers expect you to know cold. No derivations, no narrative. Every section ends with a link to the guide that explains it properly.

Notation: n = samples, d = features, k = classes/clusters/top-k, p = predicted probability, y = label, ŷ = prediction, η = learning rate, log = natural log unless stated.

Jump to: Classification · Regression · Ranking · Losses · Prob & Stats · Lin Alg & Calculus · Optimization · Algorithms · Bias-Variance · Neural Nets · LLM Numbers · System Numbers · Top 20 Traps

Classification Metrics

                  Predicted
                Neg      Pos
Actual  Neg  |  TN   |   FP   |   FP = false alarm (Type I)
        Pos  |  FN   |   TP   |   FN = miss (Type II)

• Baselines to quote: always predicting 0.5 gives Brier 0.25 and log loss log 2 ≈ 0.693. Predicting the prevalence π gives log loss -[π log π + (1-π) log(1-π)].
• Averaging (multiclass): macro = unweighted mean over classes (rare classes count); micro = pool counts (equals accuracy for single-label multiclass); weighted = by support (hides rare-class failure).

Which metric when:

• Rare-event detection (fraud, disease): PR-AUC, recall at fixed precision, precision@k.
• Threshold will be tuned later: ROC-AUC / PR-AUC (threshold-free).
• Probabilities are consumed (bidding, risk pricing, expected value): log loss + calibration plot / Brier.
• Costs are known: pick the threshold minimising C_FP·FP + C_FN·FN, not 0.5.
• Stakeholder report: precision and recall at the operating threshold, never accuracy alone.

Read more: Model Evaluation · Anomaly Detection

Regression Metrics

• SS_res = Σ(y - ŷ)², SS_tot = Σ(y - ȳ)². R² of a model that predicts ȳ is 0.
• Huber loss as a metric: MSE near zero, MAE in the tails.

Read more: Model Evaluation · Time Series

Ranking Metrics

• Position 1 discount is 1/log2(2) = 1; position 2 is 1/log2(3) ≈ 0.63; position 3 is 0.5.
• Offline ranking gains often fail to move online metrics (CTR, dwell, revenue). Always pair with an A/B test.

Read more: Model Evaluation · Recommender Systems · Search Ranking System

Loss Functions

• CrossEntropy(P, Q) = H(P) + KL(P‖Q): with fixed labels, minimising CE = minimising KL.
• Maximum likelihood view: MSE = Gaussian NLL, MAE = Laplace NLL, BCE = Bernoulli NLL.
• Forward KL KL(P‖Q) is mean-seeking (covers all modes); reverse KL KL(Q‖P) is mode-seeking.

Read more: Neural Network Training · Embeddings · Model Compression

Probability and Statistics

Core rules

• Bayes: P(A|B) = P(B|A)·P(A) / P(B), with P(B) = Σ_i P(B|A_i)·P(A_i).
• Classic: prevalence 1%, sensitivity 99%, FPR 5% → P(D|+) = 0.0099 / (0.0099 + 0.0495) ≈ 16.7%. Base rates dominate.
• E[aX + bY] = aE[X] + bE[Y] always. Var(aX + b) = a²Var(X). Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y).
• MAP with Gaussian prior on weights = L2 (ridge); Laplace prior = L1 (lasso).

• Normal: 68 / 95 / 99.7% within 1 / 2 / 3 σ. z = 1.645 (90%, two-sided), 1.96 (95%), 2.576 (99%).

Inference

• CLT: for i.i.d. samples with finite variance, x̄ ≈ N(μ, σ²/n) for large n. Standard error SE = σ/√n.
• 95% CI (mean): x̄ ± 1.96·s/√n (use t_{n-1} for small n). Proportion: p̂ ± 1.96·√(p̂(1-p̂)/n).
• A 95% CI means: 95% of intervals built this way contain the true value. Not "95% probability the value is in this interval".
• p-value: P(data at least this extreme | H0 true). It is not P(H0 | data) and says nothing about effect size.
• α = Type I rate (false positive), β = Type II rate (false negative), power = 1 - β.

A/B test sample size (per arm, two-sided):

n ≈ (z_{1-α/2} + z_{1-β})² · 2σ² / δ²
α = 0.05, power = 0.8  →  (1.96 + 0.84)² ≈ 7.85  →  n ≈ 16·σ² / δ²
binary metric: σ² = p(1-p)

• Example: baseline conversion 10%, detect +1pp absolute → 16·0.09 / 0.0001 = 14,400 per arm.
• Halving the detectable effect δ quadruples n.

Read more: Statistics and Probability · A/B Testing · Causal Inference

Linear Algebra and Calculus

Derivatives to know

Shapes and costs

• (m×n)·(n×p) → (m×p), costing m·n·p multiply-adds (2mnp FLOPs).
• Dense layer: X (batch×d_in) · W (d_in×d_out) + b (d_out) → (batch×d_out).
• Attention scores: Q (n×d_k) · Kᵀ (d_k×n) → (n×n): quadratic in sequence length.
• Normal equation: w = (XᵀX)⁻¹Xᵀy, cost O(nd² + d³); ridge: w = (XᵀX + λI)⁻¹Xᵀy (always invertible for λ > 0).

Decompositions

• Eigen: Av = λv. Symmetric matrices (covariance) have real eigenvalues and orthogonal eigenvectors.
• SVD: A = UΣVᵀ for any A (m×n); U (m×m), V (n×n) orthogonal; Σ diagonal with singular values σ_1 ≥ σ_2 ≥ ... ≥ 0.
• Truncated SVD (top k) is the best rank-k approximation in Frobenius and spectral norm (Eckart-Young).
• PCA = SVD of centred X: components are columns of V; variance along component i = σ_i² / (n-1).
• Rank = number of non-zero singular values. Condition number = σ_max / σ_min (large → numerically unstable).
• Numerical stability: compute softmax/log-sum-exp as m + log Σ e^{z_i - m} with m = max(z).

Read more: Math for ML · Dimensionality Reduction

Optimization

Weight decay vs L2

• L2 regularisation adds (λ/2)‖θ‖² to the loss → adds λθ to the gradient.
• With plain SGD this is identical to weight decay (θ ← (1 - ηλ)θ - η·g).
• With Adam it is not: the λθ term gets divided by √v̂, so parameters with large gradients are barely regularised. AdamW decouples it.
• Usually exclude biases and norm gains from weight decay.

• Gradient clipping by global norm (e.g. 1.0) guards against spikes, essential for RNNs and LLMs.
• Batch size scaling (heuristic): linear LR scaling for SGD (k× batch → k× LR, with warmup); roughly √k for Adam.

Read more: Neural Network Training · Math for ML

Algorithm Cheat Table

n = samples, d = features, k = clusters/neighbours, T = trees, L = tree depth, n_sv = support vectors.

• Tree models are invariant to monotonic feature transforms; distance, margin and gradient-based models are not.
• Default tabular pipeline: logistic regression baseline → gradient boosting → tune → only then consider deep learning.

Read more: Classical ML Overview · Ensemble Methods · Clustering · Dimensionality Reduction

Bias-Variance Diagnostics

Expected test error = bias² + variance + irreducible noise

Learning curves (error vs training-set size)

• Curves converge at a high error → high bias; more data will not help.
• Large persistent gap that shrinks with more data → high variance; more data will help.

Knobs and their direction

• Double descent: very over-parameterised nets can see test error fall again past the interpolation threshold.

Read more: Model Evaluation · Neural Network Training · Ensemble Methods

Neural Network Essentials

• Goal of all three: keep activation/gradient variance roughly constant across layers. Never initialise all weights equal (symmetry never breaks).

• All compute y = γ·(x - μ)/√(σ² + ε) + β (RMSNorm: y = γ·x/RMS(x)); they differ only in which axes give μ, σ.
• Dropout: at train, zero each unit with prob p and scale survivors by 1/(1-p) (inverted dropout); at eval, identity. Call model.eval().

Parameter counts

Attention

Attention(Q, K, V) = softmax(Q·Kᵀ / √d_k) · V

• √d_k keeps dot-product variance ≈ 1 so softmax doesn't saturate.
• Cost: O(n²·d) time, O(n²) memory for the score matrix (FlashAttention avoids materialising it, same FLOPs).
• Causal mask sets future positions to -∞ before softmax. Multi-head: h heads of size d/h, concatenated then projected.

Compute rules of thumb

• Training FLOPs ≈ 6·N·D (N params, D training tokens): 2ND forward + 4ND backward.
• Inference FLOPs ≈ 2·N per generated token (ignoring attention over the context).
• Chinchilla compute-optimal: D ≈ 20·N tokens (approximate).

Read more: Neural Network Training · Transformers · Computer Vision · Sequence Models

LLM Numbers

Memory per parameter

• Full fine-tuning with Adam, mixed precision: ≈ 16 bytes/param (2 weights + 2 grads + 4 fp32 master + 4 m + 4 v), before activations. 7B → ~112 GB.
• LoRA adds r·(d_in + d_out) params per adapted matrix; only those get gradients and optimizer state.

KV cache

KV bytes = 2 (K and V) × n_layers × n_kv_heads × d_head × seq_len × batch × bytes_per_elem

• Llama-2-7B (32 layers, 32 KV heads, d_head = 128, fp16): 2·32·32·128·2 = 512 KB/token → ~2 GB for one 4k-token sequence.
• GQA/MQA shrink n_kv_heads (Llama-3-8B: 8 KV heads → 128 KB/token). KV cache, not weights, often limits batch size.

Rules of thumb (approximate)

• English: 1 token ≈ 0.75 words ≈ 4 characters; 1,000 tokens ≈ 750 words. Code and non-English text use more tokens per word.
• Prefill is compute-bound; decode is memory-bandwidth-bound.
• Batch-1 decode ceiling: tokens/s ≈ memory bandwidth / model bytes (e.g. 14 GB model on ~2 TB/s HBM → ~140 tok/s upper bound).
• Cost/latency levers: quantisation, KV-cache reuse (prefix caching), continuous batching, speculative decoding, shorter prompts.

Read more: LLM Fundamentals · LLM Inference Optimization · Model Compression · Fine-Tuning

System Design Numbers

All values are approximate orders of magnitude; hardware varies. Use them for back-of-envelope estimates, not SLAs.

Read more: ML System Design Patterns · Model Serving · Backend System Design Guide

Top 20 Traps

1. Target leakage: a feature that is only known after the label (e.g. "refund issued" to predict fraud). Ask "would I have this at prediction time?"
2. Fitting preprocessing on all data: scalers, imputers, target encoders or feature selection fit before the split. Put them inside a Pipeline fit per fold.
3. Random split on time series: trains on the future. Use time-based splits / forward-chaining CV with a gap.
4. Entity leakage: same user, patient or session in train and test. Use GroupKFold.
5. Accuracy on imbalanced data: 99% accuracy at 1% prevalence is a constant predictor. Use PR-AUC, recall@precision, F1.
6. ROC-AUC on rare events: looks great while precision is terrible. Report PR-AUC too.
7. Oversampling (SMOTE) before splitting: synthetic copies of test points leak into training. Resample only the training fold.
8. Tuning on the test set: every peek makes it a validation set. Hold out a final test set touched once.
9. Default 0.5 threshold: choose the threshold from costs or a precision/recall target on validation data.
10. Uncalibrated probabilities: after class weights, undersampling or boosting, scores are not probabilities. Recalibrate (Platt, isotonic) or correct for the sampling rate.
11. Training-serving skew: features computed differently offline vs online. Share feature code, log served features.
12. Duplicates across splits: near-duplicate images/texts, or benchmark contamination in LLM pretraining data. Dedupe before splitting.
13. Peeking at A/B tests: stopping when p < 0.05 inflates false positives. Fix sample size up front or use sequential tests.
14. Multiple comparisons: 20 metrics at α = 0.05 gives ~1 false win by chance. Pre-register a primary metric; correct (Holm, BH).
15. Sample ratio mismatch: a 50/50 test landing 52/48 means broken assignment/logging. Check with a chi-square test before reading results.
16. Correlation ≠ causation: confounding and Simpson's paradox. Feature importance is not a causal effect.
17. Missing feature scaling: k-NN, SVM, k-means, PCA and regularised linear models all depend on scale; trees don't.
18. Forgetting model.eval(): dropout stays on and BatchNorm uses batch stats at inference.
19. Bad metric for the target: MAPE when y can be ~0, RMSE on heavy-tailed targets, R² compared across datasets.
20. Feedback loops and selection bias: training only on data your model chose to show or approve (clicks on shown items, approved loans). Log propensities, keep exploration traffic.

Read more: Feature Engineering · Time Series · Model Evaluation · A/B Testing · Model Monitoring · Glossary

Related Topics

• Glossary: every term in one or two sentences
• 2026 Interview Questions
• 2026 Interview Roadmap
• Study Pattern
• Repository Home