Yesterday we collected data into the S3 data lake. But simply throwing objects into S3 isn't enough for analysis or training. You need to know "what data is in this bucket, and under what schema?" so you can query with Athena and train with SageMaker. Managing this metadata is what data cataloging is about, and preparing data by cleaning and transforming it is what ETL means.
Today's stars are AWS Glue — a serverless data integration service. We'll cover crawlers, the data catalog, ETL Jobs, and even the no-code tool DataBrew.
Glue Data Catalog is a Hive Metastore-compatible metadata repository