The Data Catalog we created yesterday only knows "where the data is and what it looks like." But the essence of data engineering is to transform that data. Taking dirty CSV and turning it into clean Parquet, combining scattered tables into joined fact tables, converting raw logs into aggregated metrics. The way to run these transformations serverlessly is with AWS Glue ETL Job.
Internally, Glue ETL Job runs Apache Spark. In other words, Glue is a "managed Spark." Unlike EMR, where you provision and terminate clusters directly, Glue automatically provisions the Spark environment when you submit a Job and reclaims resources when it finishes. The billing unit is DPU (Data Processing Unit)—1 DPU = 4 vCPU + 16GB memory