This week laid the foundation for MLS-C01. We revisited the ML lifecycle at Specialty depth (Day 1), where to store data and in what format (Day 2), how to ingest it (Day 3), and how to label it (Day 4). Today we integrate how these pieces interlock into a single data pipeline, and consolidate the decision criteria that most often trip people up on the exam.
Connecting this week's content into a single flow looks like this.
[Sources] Click logs · IoT · Transactions ──┐
▼
[Ingest] Stream? → Kinesis (Firehose = delivery / KDS = multi-consumer & replay) (Day 3)
Batch? → Glue ETL / EMR / Batch
▼
[Store] S3 data lake (Parquet/RecordIO) (Day 2)
▼
[Label] SageMaker Ground Truth (workforce + active learning + consensus) (Day 4)
▼
[Features] Feature engineering with Glue/Processing → Feature Store
▼
[Training I/O] S3 (Pipe/File/FastFile) | EFS | FSx for Lustre (Day 2)
▼
[Model] Train → Evaluate (tie to business metrics) → Deploy → Monitor (Day 1)
└──── loop back on drift ────┘
💡 Related theory: The principles that run consistently through this entire pipeline are reproducibility and preventing training-serving skew. Fix ingestion, cleaning, and feature logic in code, version your data, and use identical transformations in training and inference. Features improvised in a notebook or data of unknown provenance quietly break your model in production.
Compressing the choices that most often split exam takers into one-line rules.
Storage input mode (Day 2)
| Situation | Choice |
|---|---|
| Large, sequential, fast start | Pipe mode |
| Small data, random access | File mode |
| Large but partial/random reads only | FastFile mode |
| Repeated, high-throughput sharing | FSx for Lustre |
| General-purpose shared file system | EFS |
Data format (Day 2)
Click a choice to reveal the answer and explanation.
Question 1
You are comparing the performance of a fraud detection model where only 0.1% of transactions are fraudulent and false negatives are costly. What is the most honest single metric?
Question 2
You are distributing training of hundreds of GB of data across multiple instances, minimizing GPU idle time while having each instance read only a different slice of the data. What do you use?
Question 3
You need to label medical data containing PII while also reducing labeling labor costs. What is the most appropriate combination?
Question 4
A single event stream must be used independently for a real-time dashboard, a fraud model, and later reprocessing, and replay is needed after failures. Which ingestion service fits?
Question 5
When solving a scenario question with multiple simultaneous constraints (sensitive data, extreme imbalance, multi-consumer stream), what is the correct approach?
| Situation | Choice |
|---|---|
| Structured data analytics/ETL, only some columns | Parquet (columnar) |
| Large-scale training with SageMaker built-in algorithms | RecordIO-protobuf |
| Millions of small files | Consolidate into large bundles via sharding |
Ingestion service (Day 3)
| Situation | Choice |
|---|---|
| Simple no-code delivery to S3 etc. | Kinesis Data Firehose |
| Multiple consumers, reprocessing, custom processing | Kinesis Data Streams |
| Real-time stream aggregation & anomaly detection | Managed Service for Flink |
| Schema inference & catalog | Glue Crawler |
| Serverless Spark ETL | Glue ETL Job |
Labeling (Day 4)
| Situation | Choice |
|---|---|
| Sensitive or regulated data | Private or Vendor workforce |
| Public, large-scale, low-cost | Mechanical Turk |
| Reducing labeling cost | Active learning (automated labeling) |
| Reducing random errors | Consensus (multiple labelers + consolidation) |
# Classification metric decision tree (pseudocode)
if classes_are_severely_imbalanced:
if false_negative_cost_is_high (fraud, disease): → prioritize recall, compare with PR-AUC
elif false_positive_cost_is_high (spam, marketing): → prioritize precision
else: → PR-AUC
else:
if threshold_undecided: → ROC-AUC
else: → F1 / accuracy💡 Related theory: Always beware the single-metric trap. On imbalanced data, 99.9% accuracy can be an "all negative" model. If you remember the two-stage structure — offline metrics as the deployment gate, online business metrics (A/B testing) as the final verdict — you have captured the core of Day 1.
Scenario: A global e-commerce company wants to detect fraudulent transactions from a real-time clickstream. Historical fraud labels are very scarce, and only 0.1% of transactions are fraudulent. The data contains PII.
The correct step-by-step flow:
This single scenario contains every concept from Week 1. If you can explain the reasoning behind each decision in words, you have digested the week.
💡 Related theory: In scenario questions where multiple concepts collide, "constraints" come first. PII forces the workforce, extreme imbalance forces the metric, and the multi-consumer requirement forces KDS. The correct answer is not the fanciest technology among the options, but the choice that satisfies all constraints simultaneously.
Week 2 is Data Engineering 2: large-scale processing with EMR and Spark, deeper data transformation and cleaning, handling missing values, outliers, and imbalanced data, and the real techniques of feature engineering. It is the week we dig deep into the "Features" stage of the pipeline we consolidated today.