Day 3 - Overfitting/Underfitting: Diagnosis and Regularization/Data Augmentation

Even with good hyperparameters from tuning, if a model fits training well but validates poorly, it's useless. Today covers generalization failure's two sides — overfitting and underfitting — how to diagnose, and how to fix each with regularization (L1/L2/dropout/early stopping) and data augmentation. Tests ask about learning curve patterns, bias-variance, matching fixes.

Bias and Variance, Generalization

Generalization error decomposes into bias and variance.

• High bias = underfitting: Model too simple to capture training patterns. Both training and validation error high
• High variance = overfitting: Model memorizes noise. Training error low, validation error high (big gap)