Day 3 - AWS ML Stack at a Glance

Now that you've identified the ML problem, it's time to choose your tools. AWS has dozens of ML-related services, which feels overwhelming at first. But in reality, they're organized cleanly into three layers. Answer the question "Will you build a model from scratch, call pre-trained models via API, or manage bare infrastructure?" and you'll know which layer to use.

Today, we'll survey this 3-layer stack — AI Services, ML Platform (SageMaker), and Infrastructure — see when ML engineers choose each layer, and learn how to identify in exam scenarios "which tool should we use?"

AWS ML Stack: The 3-Layer Structure

\
┌─────────────────────────────────────────────────┐ │ Top: AI Services (Pre-trained models, API calls only)│ │ Rekognition, Comprehend, Transcribe, │ │ Translate, Polly, Textract, Forecast, Bedrock │ ├─────────────────────────────────────────────────┤ │ Middle: ML Platform (Train & deploy directly) │ │ Amazon SageMaker (AI) │ ├─────────────────────────────────────────────────┤ │ Bottom: ML Infrastructure (Direct compute mgmt) │ │ EC2(GPU), Inferentia, Trainium, EKS, ECS, FSx │ └─────────────────────────────────────────────────┘ \\

The selection principle is the tradeoff between abstraction level and control. Go higher and it's faster and easier, but less control. Go lower and you're more free, but you handle more directly. ML engineers choose: "Top for standard tasks that don't need ML expertise, Middle for custom models, Bottom for extreme performance/cost optimization."

💡 Related Theory: This is the ML version of the shared responsibility model from Day 1. AI Services have AWS handling model training and infrastructure; the customer calls APIs and provides data. SageMaker has the customer handling model code and data while AWS manages infrastructure. Self-managed EC2 puts almost everything on the customer. As abstraction rises, the responsibility boundary moves up.

Top Layer: AI Services — ML Capabilities Without Models

AI Services use pre-trained models with a single API call. You need almost no ML knowledge. Memorizing common mappings helps exam scenario questions.

Summary

Two key takeaways today. First, the AWS ML stack is AI Services (API calls) → SageMaker (train directly) → Infrastructure (direct compute), with tradeoffs between abstraction and control. Second, at the infrastructure layer, chips are split: Trainium for training, Inferentia for inference.

Tomorrow we'll dive deep into the heart of this stack — SageMaker itself.

📝 Practice Questions

Click a choice to reveal the answer and explanation.

Question 1

For the standard requirement "analyze positive/negative sentiment in English customer review text," which AWS service is most appropriate?

• ATrain a sentiment analysis model directly in SageMaker
• BCall Amazon Comprehend's sentiment analysis API
• CBuild a custom model on EC2 GPU instances
• DAmazon Polly

Question 2

Which correctly describes the characteristic as you move up the 3-layer AWS ML stack (toward AI Services)?

• AControl increases and more needs direct management
• BAbstraction rises making it faster and easier, but control decreases
• CCost is always higher
• DMore ML expertise is required

Question 3

An ML engineer minimizing inference cost for massive traffic should consider which AWS custom chip?

• AAWS Trainium
• BAWS Inferentia
• CFSx for Lustre
• DElastic Block Store

Question 4

What's the defining difference between SageMaker and AI Services (Rekognition, Comprehend, etc.)?

• ASageMaker only allows API calls
• BSageMaker trains and tunes custom models directly on your data and deploys them
• CSageMaker doesn't support model training
• DAI Services always provide more control than SageMaker

Question 5

What makes it rational to design training and inference chips separately?

• ATraining and inference have the same computational characteristics
• BTraining does forward+backward passes for throughput, inference does forward passes only for low latency — different characteristics justify separate chips
• CInference happens far less frequently than training
• DTo equalize chip prices