Day 2 - Hyperparameter Tuning (AMT): Bayesian, Random, Hyperband

Yesterday we learned to run one training job well. Today, run that training automatically many times to find optimal hyperparameters via SageMaker Automatic Model Tuning (AMT). Tests ask about search strategies (Bayesian/random/grid/Hyperband), early stopping, warm start, objective metrics.

Hyperparameter vs Parameter

First, terminology:

• Parameter: Learned from data during training (e.g., neural net weights, linear model coefficients)
• Hyperparameter: Human-set before training (e.g., learning rate, max_depth, tree count, regularization strength)

AMT explores hyperparameter space to find the combination with best validation metric.