Based on the SAP-C02 exam guideReport an error
SAP-C02 Domain 4, "Continuous Improvement for Existing Solutions," is the second-largest at 25% of the exam. Its essence is "do you improve a system that is already running, incrementally, while operating it," and its roots are in SRE (Site Reliability Engineering). SRE, established by Google in 2003 on Ben Treynor Sloss's team, is a paradigm of "treat operations as a software engineering problem (automate Toil with code) + manage reliability with measurable indicators (SLI/SLO/SLA + Error Budget)." For this, AWS provides tools across six areas: observability, automation, deployment, cost, resilience, and security.
Today we organize Domain 4 together with SRE theory as one scenario mapping - not a simple tool table: if you understand the principle of "why this way of operating is the right answer," you can solve variant questions.
Modern observability is defined by the three pillars of Metrics, Logs, and Traces (this classification was laid out by Peter Bourgon in 2017 and became an industry standard). Metrics answer "what and how much" (aggregate numbers), logs answer "what happened" (discrete events), and traces answer "where did it slow down" (the path of a request across services).
| Pillar | AWS tools | Question it answers |
|---|---|---|
| Metrics | CloudWatch Metrics, Alarms | What are the CPU, latency, and error rate |
| Logs | CloudWatch Logs, Insights | What error occurred |
| Traces | X-Ray, ADOT | In which service did the request slow down |
| Tool | Use |
|---|---|
| CloudWatch Metrics | Time-series metrics, 1-second to 1-minute resolution |
| CloudWatch Logs Insights | Ad-hoc log queries (billed per GB scanned) |
| CloudWatch Synthetics | Synthetic user monitoring (canary) |
| CloudWatch RUM | Real user monitoring (Core Web Vitals, JS errors) |
| Application Insights | Automatic anomaly detection and root cause |
| Container/Lambda Insights | Container and Lambda performance metrics |
SRE's reliability management stands on the math of SLI/SLO/SLA + Error Budget. The SLI (Indicator, the measured value: for example availability 99.95%), the SLO (Objective, the target: 99.9%), and the SLA (Agreement, the contract: refund if below 99.5%). The key is Error Budget = 100% − SLO - if the SLO is 99.9%, about 43 minutes of downtime per month is the "allowed budget." While budget remains you deploy new features quickly, and when it is used up you stop deploying and focus on stabilization. This is SRE's core innovation, mediating the conflict between development (speed) and operations (stability) with numbers. On the exam, the thinking behind "quantifying reliability targets, balancing deployment speed versus stability" comes from this model.
🔍 Going deeper: CloudWatch Logs Insights does a parallel scan at query execution time without indexing, so it is fast for ad-hoc analysis, but it is billed per GB of data scanned, so broad, repeated searches become expensive. Alternatives compared: S3 + Athena (cheap storage and SQL analysis, serverless), OpenSearch (real-time full-text search and dashboards via an inverted index, but the burden of operating a cluster). Pattern: short-term operational debugging with Logs Insights, long-term retention and large-scale analysis with S3+Athena, real-time search and Kibana dashboards with OpenSearch. "Low-cost retention of years of logs + occasional SQL analysis" goes to S3+Athena, and "real-time log search and visualization" goes to OpenSearch.
| Tool | Standard |
|---|---|
| X-Ray | AWS proprietary SDK |
| ADOT (AWS Distro for OpenTelemetry) | OpenTelemetry (CNCF) standard |
| ServiceLens | Integrated X-Ray + CloudWatch view |
OpenTelemetry (OTel) is a vendor-neutral standard for distributed tracing, metrics, and logs created in 2019 when the CNCF merged OpenTracing + OpenCensus. Its conceptual roots are in Google's Dapper paper (2010) - the idea of propagating a trace ID with a request in a large distributed system to reconstruct the path across services. AWS keeps its own X-Ray while providing OTel compatibility with ADOT (2020). If you need multi-cloud and avoidance of vendor lock-in, it is ADOT (send to AWS, GCP, Azure, and Datadog with the same SDK), and if it is AWS-only and simple integration, it is X-Ray. "Standard tracing without vendor lock-in, multi-cloud" is the signal for ADOT.
The SRE principle: automate repetitive, manual, non-scalable operational work (Toil) so that engineers can focus on improvement. Google SRE sets the norm "do not spend more than 50% of the team's time on Toil."
| Feature | Use |
|---|---|
| Patch Manager | Automatic OS patch deployment (rules per group) |
| Session Manager | Connect to EC2 without SSH keys or a Bastion (audit logging) |
| Run Command | Run commands in bulk on many EC2 instances |
| Automation Document | Automate complex operational workflows |
| Incident Manager | Integrated pager escalation, runbooks, and communication |
🎯 Scenario: "Security patches must be applied monthly to 500 EC2 instances, but security policy forbids SSH access and Bastion hosts. Patches must be rolled out gradually by group during set time windows to avoid business impact. What is the design?" - Answer: SSM Patch Manager (patch baselines and patch groups) + Maintenance Window (time window control) + Session Manager (console debugging without SSH). Patch Manager rolls out gradually by patch group, Maintenance Window controls the time such as "weekends 2-4 AM," and when debugging is needed, Session Manager connects without SSH keys or a Bastion (every session is audit-logged to CloudTrail and S3). The burden of managing Bastions and SSH keys drops to zero. "SSH forbidden + patch automation + time control" is answered directly by this set of 3.
The AWS S3 us-east-1 major outage of 2017 (about 4 hours) shows the double edge of automation tools. While debugging the billing system, an engineer made a typo in a playbook command and took down far more S3 servers than intended, restarting the index and placement subsystems and causing a cascading failure. The lessons are (1) put safeguards on manual commands (limit blast radius, confirmation steps), (2) roll out large operations gradually (canary), and (3) make runbooks into code (SSM Automation) to reduce human typos. This is the historical reason why "minimal human intervention, automating repetitive operations" on the exam leads to SSM Automation and EventBridge.
[CloudWatch Alarm] → [EventBridge] → [SNS / Lambda / SSM Automation / Incident Manager]
(A hypothetical case) A SaaS company: DynamoDB throttle alarm → EventBridge → Lambda automatically increases capacity → Slack notification → root cause analysis on the next business day. It is a self-healing pattern where recovery happens automatically without a person being woken in the middle of the night. This is an example of "operations as code," which SRE aims for.
| Strategy | Behavior | Suitable workload |
|---|---|---|
| Rolling | Replace some instances at a time | Small scale, fast rollback not needed |
| Blue/Green | Create a whole new environment → switch all at once | Mission critical, instant rollback |
| Canary | Traffic 1%, 10%, 100% gradually | Gradual verification |
| Linear | Increase by N% every N minutes | Time-based gradual |
CodeDeploy Lambda Canary increases traffic gradually starting from 1% while monitoring CloudWatch alarms, and automatically rolls back when an alarm fires. The key pitfall is that two versions run simultaneously during Canary and Blue/Green, so the DB schema must be backward-compatible - if you delete or rename a column, the old version breaks. So you use the "expand and contract" pattern (first add the column → both versions work → remove the old column later). "Zero-downtime deployment + instant rollback" is Blue/Green, and "Lambda gradual traffic + automatic rollback" is CodeDeploy Canary.
| Tool | Characteristics |
|---|---|
| CloudFormation / StackSets | AWS native, StackSets deploys across the whole Org |
| CDK | TS/Python → generates CFN |
| SAM | CFN extension specifically for serverless |
| Terraform / Pulumi | Multi-cloud |
| Tool | Use |
|---|---|
| Compute Optimizer | EC2/EBS/Lambda right-size recommendations (ML) |
| Cost Explorer / CUR | Cost visualization and forecasting / detailed billing → S3 |
| Budgets / Budgets Action | Budget alerts / automatic actions on overrun (IAM deny, etc.) |
| Cost Anomaly Detection | ML-based detection of abnormal costs |
| Trusted Advisor | Checks across 5 areas (Cost, Perf, Security, Fault Tolerance, Limits) |
FinOps (Financial Operations) is a cloud cost operations culture established by the FinOps Foundation in 2019, where the core is "engineers make decisions with cost awareness." A 3-phase cycle - Inform (visibility: Cost Explorer, CUR, tagging) → Optimize (optimization: right-size, SP/Spot, Compute Optimizer) → Operate (operations: automating budgets and anomaly detection). The heart of SP (Savings Plans) math: a Compute SP gives up to ~66% discount through an hourly commitment ($/h) spanning EC2, Fargate, and Lambda while staying flexible on instance family and Region changes, and an EC2 Instance SP is cheaper but fixes the family. "A unified discount across Lambda, Fargate, and EC2 + flexibility" is Compute SP.
Chaos engineering originated from Chaos Monkey (randomly killing instances to force verification of resilience), which Netflix created in 2010 while migrating to AWS, and it expanded into the Simian Army in 2011. The core principle is "failures are inevitable, so inject them deliberately in a controlled environment to find weaknesses in advance." AWS FIS provides this as a managed service, and the Stop Condition (automatically stops the experiment when a CloudWatch alarm exceeds a threshold) is the safety net - it stops the experiment before it spreads into a real failure. "Quarterly Game Day, deliberate fault injection + automatic safety stop" is answered directly by FIS + Stop Condition.
⚠️ Pitfall: Do not confuse the cost and RTO trade-off of DR strategies. Backup&Restore (cheapest, RTO in hours) < Pilot Light (only the core on) < Warm Standby (scaled-down always-on) < Multi-Site Active-Active (most expensive, RTO in minutes/seconds). "RTO in minutes + cost compromise" is Warm Standby, and "RTO near 0 + cost no object" is Active-Active. Pitfall: a cold standby (Backup&Restore) is unverified at the moment of failover and fails easily - which is why Warm or better, which receives some traffic even in normal times, is recommended.
Layer 1 KMS, CloudHSM; Layer 2 Macie, GuardDuty, Inspector; Layer 3 Security Hub, Detective, Audit Manager; Layer 4 WAF, Shield, Firewall Manager, Network Firewall, DNS Firewall.
The 4 Golden Signals of the Google SRE Book (the 4 to measure first when monitoring a service):
| Signal | Meaning | AWS tools |
|---|---|---|
| Latency | Response time | CloudWatch Metrics, X-Ray |
| Traffic | Throughput | ELB, API GW metrics |
| Errors | Failure rate | CloudWatch, RUM |
| Saturation | Resource saturation | CloudWatch, Container Insights |
| Keyword | Answer |
|---|---|
| "EC2 right-size recommendations" | Compute Optimizer |
| "Automatically stop on budget overrun" | Budgets Action |
| "Automatic incident pager, runbooks" | Incident Manager |
| "Zero-downtime ECS deployment, instant rollback" | CodeDeploy Blue/Green |
| "Lambda gradual traffic, automatic rollback" | CodeDeploy Canary |
| "SSH forbidden + patch automation + time control" | SSM Patch Manager + Maintenance Window + Session Manager |
| "Quarterly chaos, automatic safety stop" | FIS + Stop Condition |
| "Automatically assess workload RTO gap" | Resilience Hub |
| "ML detection of abnormal costs" | Cost Anomaly Detection |
| "Unified discount for Lambda, Fargate, EC2" | Compute Savings Plans |
| "Distributed tracing without vendor lock-in" | ADOT (OpenTelemetry) |
| "Years of logs at low cost + SQL analysis" | S3 + Athena |
| "Real-time log search, dashboards" | OpenSearch |
| "Real User monitoring" | CloudWatch RUM |
| "Synthetic transaction monitoring" | CloudWatch Synthetics |
| "Human-decision failover" | Route 53 ARC Routing Control |
Domain 4 is the cloud application of SRE. The core cycle: measure (three pillars of observability, SLI/SLO, Error Budget) → automate (eliminate Toil, SSM and EventBridge self-healing) → deploy gradually (Blue/Green, Canary, schema compatibility) → continuously optimize cost, resilience, and security (FinOps, chaos engineering). On the exam, when you see the keywords "minimal operational burden, minimal human intervention, automatic recovery," the answer is almost always automation such as SSM, EventBridge, and Incident Manager. The historical incidents (the 2017 S3 typo outage, Netflix Chaos Monkey) explain why "runbooks as code and controlled fault injection" are the right answers.
Tomorrow (Day 80) is the wrap-up of all 16 weeks + a practice exam + D-Day strategy.
Click a choice to reveal the answer and explanation.
Question 1
Security patches must be applied monthly to 500 EC2 instances, but security policy forbids SSH and Bastions. Patches must be rolled out gradually by group during set time windows. What is the most suitable configuration?
Question 2
You want to deploy Lambda function v2 by increasing traffic gradually from 1%, and automatically roll back to v1 if a CloudWatch alarm fires. What is the most suitable method?
Question 3
A service sets its availability SLO at 99.9%, and wants to deploy new features quickly only while exceeding this target, but stop deploying and focus on stabilization once the budget is exhausted. What are the name of this operating model and its core metric?
Question 4
Every quarter you want to deliberately inject failures to verify system resilience, but the experiment must stop automatically if the real user impact exceeds a threshold. What is the most suitable tool and configuration?
Question 5
You want to retain years of application logs at low cost while occasionally running ad-hoc analysis with SQL. You want to avoid the burden of operating a cluster. What is the most suitable combination?
Question 6
You want to be automatically detected and alerted when costs spike abnormally compared with the usual pattern. It must also catch new anomaly patterns for which you cannot know an absolute threshold in advance. What is the most suitable tool?
Question 7
In a multi-cloud environment (AWS + on-premises K8s), you want to collect distributed traces with the same instrumentation SDK to avoid vendor lock-in. What is the most suitable choice?
Question 8
During a Region failure, instead of automatic health-check-based failover, you want the operations team to assess the situation and switch traffic to another Region manually (but with high reliability). What is the most suitable service?