Yesterday we read data via Athena and Redshift. But how you store the same data can make query costs differ by 50x and training speed vary by orders of magnitude. Storage isn't about "where to put it" but "how to pre-arrange it so the reader can access it fast and cheap."
Today we explore the three pillars of ML data storage — partitioning, file format optimization, and preparing data to hand off to SageMaker training. In MLA-C01 Domain 1, storage strategy consistently appears in scenarios testing cost and performance.
Partitioning is splitting data into folders by a specific key (usually date)