Day 5 - Week 1 Comprehensive Review — ML Fundamentals & AWS Stack

This week we surveyed ML basics and how to implement them on AWS. Today we tie the scattered pieces together. MLA-C01 tests not isolated knowledge but "which tool at which stage for this situation?" — so the most efficient way to learn is to layer concepts and services on the ML lifecycle.

Four dimensions to review today: ① ML lifecycle, ② problem types and metrics, ③ SageMaker essentials, ④ AWS AI Services. We'll organize each by "when do you pick what?" decision criteria.

ML Lifecycle: The Full Picture

ML projects don't train once and end — they cycle. The four domains of MLA-C01 map directly to this lifecycle.

[1. Data Prep] → [2. Model Dev] → [3. Deploy·Serve] → [4. Monitor·Maintain]
  Collect/Store/Transform   Train/Tune/Evaluate   Pick Endpoint   Drift/Retrain
  (Domain 1, 28%)           (Domain 2, 26%)       (Domain 3, 22%) (Domain 4, 24%)
        ↑                                                              |
        └──────────────── Retrain Trigger ──────────────────────────┘

The key is that last arrow. Deployed models degrade over time as data distribution drifts, then you cycle back to data prep. ML engineers' job is to automate and stabilize this cycle.

💡 Related Theory: This cycle is standardized as CRISP-DM (Cross-Industry Standard Process for Data Mining). Established in the 1990s, it defines 6 iterative stages: business understanding → data understanding → data preparation → modeling → evaluation → deployment. Modern MLOps tightly couples this to "post-deployment monitoring and automated retraining." Exam scenarios saying "model performance declines over time" always point to the monitoring/retrain stage in this cycle.

Problem Types and Metrics

What problem a model solves dictates its metrics. Pick the wrong metric and you build a "99% accurate but useless model."

Also review supervised/unsupervised/reinforcement learning. Supervised learns from labeled data (classification·regression). Unsupervised finds structure without labels (clustering, dimensionality reduction, anomaly detection). Reinforcement learns policies from rewards.

from sklearn.metrics import precision_score, recall_score, f1_score
 
# With imbalanced data (0.1% fraud), accuracy is a trap — look at precision/recall
precision = precision_score(y_true, y_pred)   # Share of positive predictions that are true
recall = recall_score(y_true, y_pred)         # Share of actual positives we caught
f1 = f1_score(y_true, y_pred)                 # Harmonic mean of both

💡 Related Theory: The Precision-Recall tradeoff comes from asymmetric costs. For "missing is catastrophic" problems like fraud detection, prioritize Recall (catching misses). For "false alarms are painful" like spam filters, prioritize Precision (reducing false alarms). With extreme class imbalance, Accuracy is meaningless — predicting all as the majority class hits 99%. That's why imbalanced problems use F1 or AUC-PR as summary metrics.

⚠️ Gotcha: Don't confuse RMSE and MAE in regression. RMSE squares large errors, penalizing them harder, so it's sensitive to outliers. MAE treats all errors equally, more robust. "Lots of outliers and I need to specially penalize large errors" → RMSE. "Reduce outlier impact" → MAE.

SageMaker Essentials Organized

SageMaker, which we explored in the second half of this week, is a toolkit covering the entire ML lifecycle. By stage:

Key exam points: training happens in ephemeral containers (auto-terminates, spot cuts cost), and inference options split by traffic pattern.

import sagemaker
from sagemaker.estimator import Estimator
 
session = sagemaker.Session()
role = sagemaker.get_execution_role()
 
# Train tabular classification with built-in XGBoost
estimator = Estimator(
    image_uri=sagemaker.image_uris.retrieve("xgboost", session.boto_region_name, "1.7-1"),
    role=role,
    instance_count=1,
    instance_type="ml.m5.xlarge",
    use_spot_instances=True,   # Cut training costs
)
estimator.fit({"train": "s3://my-bucket/train/"})

🔍 Deeper Dive: Memorize inference option choice in one line. "Thousands of requests/second, low latency" → Real-time endpoint. "Sporadic, no idle cost" → Serverless. "Bulk batch scoring" → Batch transform. "Large payload, long processing" → Asynchronous. If built-in doesn't fit, use script mode (your code + AWS framework container) or custom container.

AWS AI Services: Pre-Trained Models Without Training

Not every ML problem needs direct training. AWS offers pre-trained managed AI Services via API. "Common task, want it fast" — this is the answer.

Decision criterion: abstraction level. AI Services (API call, no training) → SageMaker (direct train/deploy, flexible) → EC2 direct (max control, max burden). "Common problem + speed" → AI Service. "Custom model on my data" → SageMaker.

💡 Related Theory: This hierarchy is the cloud version of "build vs buy." AI Services borrow pre-trained models AWS trained on massive datasets (buy). SageMaker builds custom on your data (build). The ML engineer's decision point: "Is my problem general or domain-specific?" General NLP/vision tasks are faster, more accurate, and less operational burden via AI Services. Domain-specialized data (medical imaging, manufacturing defects) needs direct training.

Summary

Week 1's big picture: ML cycles through the lifecycle (prep → dev → deploy → monitor → retrain), each problem type needs different metrics (watch for accuracy traps in imbalance), the whole lifecycle is covered by SageMaker, and common tasks attach fast via AI Services without training. The core exam skill is choosing abstraction level (AI Services → SageMaker → EC2) by problem generality.

Next week (Week 2) we dive deep into the lifecycle's first stage: data collection and storage — S3, Kinesis, Glue, Athena.