Week 12: final review, four domains in integrated flow. Today, Domains 1 (data eng) & 2 (EDA) — data collected, stored, transformed, cleaned, featured into training-ready form. The pipeline data → lake → transform → clean → visualize → ML-ready.
[Raw sources]
│
├─ 1) COLLECT/LOAD (Domain 1)
│ Streaming → Kinesis (Streams/Firehose)
│ Batch → S3 + Glue catalog
│ Migration → DMS, DataSync
│
├─ 2) STORE (Domain 1)
│ Lake = S3 (de facto standard)
│ Query = Athena (serverless), Redshift (warehouse)
│
├─ 3) TRANSFORM/ETL (Domain 1)
│ Glue (serverless Spark) / EMR (clusters)
│
├─ 4) CLEAN (Domain 2)
│ Mi