Week 4 focuses on the second axis of exploratory data analysis—handling high-dimensional data. As the number of features grows, models appear to have more information, but in practice, data becomes sparse in space and distance-based algorithms collapse due to the curse of dimensionality.
Today, we explore why this curse is a problem and two leading mitigation techniques: PCA (Principal Component Analysis) and t-SNE. MLS-C01 exams typically ask "when do you use PCA, and why is t-SNE visualization-only?"
As dimensionality increases, the volume of space that must be filled by a fixed sample size grows exponentially