Based on the SAP-C02 exam guideReport an error
In 2010 when Netflix migrated from DVD-by-mail to streaming, from own datacenters to AWS, engineers faced a terrifying truth: in cloud, instances die without warning anytime. Their response defied intuition. Not "fear failure so avoid it" but "intentionally, frequently, during business hours, cause failures and constantly prove the system survives anyway." This birthed Chaos Monkey—randomly killing production instances—and created the field of Chaos Engineering.
In SAP-C02, resilience doesn't end with "Multi-AZ enabled." Pro perspective is active validation: auto-evaluate resilience against RTO/RPO goals (Resilience Hub), safely inject intentional failures (FIS), precisely control failover traffic (Route 53 ARC). Today we decompose chaos engineering's origins, FIS Stop Condition's safety engineering, and tool choice per DR location (DRS vs MGN).
Resilience design's first step: objectively know "does my workload actually meet defined RTO/RPO?" Instead of humans guessing architecture diagrams, AWS Resilience Hub analyzes automatically.
Operation is clear: (1) register workload (CloudFormation stack, Terraform state, Resource Group auto-discovers resources), (2) input RTO/RPO targets as policy, (3) Resilience Hub analyzes whether current architecture meets targets, identifying policy violations and gaps, (4) offers concrete improvement recommendations like "add Multi-AZ for RTO 30 seconds, increase backup frequency for RPO 5 minutes," (5) auto-generates FIS experiment templates for validation.
💡 Related Theory: Resilience Hub's most powerful use is CI/CD pipeline integration. Every PR calculates resilience score—making resilience a gate like code quality—"this change worsens RTO so merge blocked" becomes possible. This Resilience as Code paradigm prevents regression like test coverage or security scanning. WA Tool is qualitative interviews; Resilience Hub is quantitative, automated RTO/RPO evaluation. In exams, "auto-identify RTO/RPO gap + concrete recommendations + CI/CD" is Resilience Hub.
🔍 Deeper Dive: Resilience Hub and WA Tool overlap but differ. WA Tool qualitatively evaluates all six pillars via questions; Resilience Hub deeply, automatically measures only Reliability's RTO/RPO. Hub discovers actual resources and generates FIS experiments; WA Tool relies on human answers. In exams, "six-pillar structured assessment" is WA Tool; "quantitative RTO/RPO gap + auto experiment generation" is Resilience Hub.
FIS is Netflix's Chaos Monkey idea that AWS commoditized as managed service in 2021. Core: Experiment Template—combination of Targets (which resources) and Actions (what failure).
Rich failure types injectable:
💡 Related Theory: Chaos engineering isn't simple "failure testing" but follows scientific experiment methodology. Proper procedure: (1) define steady state with measurable metric ("1000 orders/sec"), (2) hypothesize "this metric persists even under failure," (3) inject actual failure, (4) verify hypothesis—metric sustained means resilient, collapsed means weakness found. This brings hypothesis-verification loop from experimental science into software engineering; "tests confirm known, chaos experiments discover unknown" summarizes it. In exams, FIS framed as "discover resilience weaknesses unknown before production."
📚 Case Study: Value of chaos engineering proven inversely by 2012 Netflix Christmas Eve major outage. AWS ELB us-east-1 failure stopped Netflix streaming during peak season for hours. Netflix then evolved beyond Chaos Monkey into "Chaos Kong"—simulating entire AWS region death—and regularly rehearsed regional failover. Result: subsequent real AWS region failures were nearly imperceptibly absorbed. Lesson: failover works only as well as practiced beforehand. In exams, "validate region-failure resilience regularly before production" signals FIS-based Game Day.
Chaos engineering's greatest risk is obvious: intentional failure spirals out of control into real major incident. FIS's decisive superiority over self-written Chaos Monkey is Stop Condition—safety mechanism.
Stop Condition connects to one or more CloudWatch Alarms. If during experiment the alarm triggers (e.g., "error rate >5%", "latency >1 sec"), FIS immediately stops all failure injections and restores normal state. Pre-define "up to here is okay, beyond this line we stop"—safety boundary.
💡 Related Theory: Stop Condition implements safety engineering's automatic safety stop (fail-safe/dead-man's switch) in cloud. Same philosophy as reactor SCRAM, elevator emergency brake, industrial machine safety cutoffs—"detect danger signal, don't await human judgment, auto-revert to safest state." Without it, incident can grow before experimenter reacts. FIS combines IAM role separation (experiment affects only defined resources) and blast radius pre-definition for multi-layer safety. In exams, "chaos experiment doesn't become real disaster" is Stop Condition signal.
⚠️ Pitfall: Understanding FIS only as "failure tool" sets exam traps. FIS's true value is "controlled" failure surrounded by Stop Condition, blast radius, IAM separation. Choices like "Lambda directly terminate EC2," "Systems Manager manual failure injection" lack safety mechanisms—not proper chaos engineering. In exams, when "intentional failure + alarm auto-stop" both appear, FIS with built-in Stop Condition is only answer.
Beyond resilience validation, actual DR environment building presents AWS's two block-level replication services. Both descended from same replication engine (formerly CloudEndure) but opposite purposes.
| Item | MGN (Application Migration) | DRS (Elastic Disaster Recovery) |
|---|---|---|
| Purpose | One-time migration | Continuous DR |
| Source | On-prem/other cloud → AWS | On-prem/other cloud/AWS Region → AWS Region |
| Cutover | Once (end after migration) | Repeatable (regular drills) |
| Replication | Until migration complete | 24/7 continuous |
| RPO/RTO | - | RPO seconds, RTO minutes |
| Cost | Stop after source terminated | Sustained DR environment |
Key distinction: "move and done (MGN) vs continuously maintain (DRS)?" On-premise 200 servers permanently to AWS = MGN (7R Rehost); on-premise stays, AWS maintains 24-hour DR standby = DRS.
🔍 Deeper Dive: DRS's strength is block-level continuous replication + repeatable drills. Can spin up DR site, test (drill), return to standby—validate "does DR actually work?" regularly without production impact. DRS keeps replicated data in cheap staging area normally, converts to instances only on failover (similar to Pilot Light cost efficiency). In exams, "on-prem + AWS sustained DR + regular drills" = DRS; "on-prem → AWS permanent" = MGN—clear split.
Auto health-check failover is fast, but for mission-critical systems, false positive (misinterpreting transient network flutter as failure) causing unnecessary failover can be riskier. Route 53 Application Recovery Controller (ARC) refines this control.
📚 Case Study: The 2021-12-07 AWS us-east-1 massive outage (~7 hours, API Gateway, console, many services affected) started from unexpected auto-scaling of internal network device. Some firms experienced auto health check ping-ponging between healthy/unhealthy—traffic instability. With ARC Routing Control, operators could explicitly decide "now switch to us-west-2" for more controlled response. Outage taught "keep control plane and data plane separate"—ARC Routing Control data plane distributed across 5 regions to work even when us-east-1 dies. In exams, "mission-critical auto failover false positive concern" signals ARC Routing Control.
🎯 Scenario: "Fintech runs us-east-1/us-west-2 active-active. Auto health check misinterprets partial failure causing traffic instability. Operators want to carefully decide failover themselves. Use what?" — Answer: Route 53 ARC Routing Control. Explicit toggle separate from auto health check—humans control. Simple Route 53 Health Check (automatic) has false positive problem. "Isolate AZ only" = Zonal Shift; "continuously verify DR readiness" = Readiness Check—functions split within ARC.
Resilience completes via validation, not design. Resilience Hub auto-identifies RTO/RPO gaps, generates concrete recommendations and FIS experiments, integrates CI/CD to gate resilience like code quality. FIS implements Netflix's chaos engineering as managed service with Stop Condition, blast radius, IAM separation safety engineering preventing intentional failures from becoming real disasters. DR location build splits by purpose—MGN (one-time migration) vs DRS (sustained DR)—and failover traffic control refines via Route 53 ARC's Routing Control (explicit human decision), Zonal Shift (AZ isolation), Readiness Check (prep validation).
SAP exam frequent mappings: (1) "Auto-identify RTO/RPO gap + recommendations + CI/CD" → Resilience Hub, (2) "Intentional failure during operation + alarm auto-stop" → FIS + Stop Condition, (3) "On-prem → AWS permanent migration" → MGN, (4) "On-prem + AWS sustained DR + regular drills" → DRS, (5) "Auto failover false positive concern, human explicit decision" → ARC Routing Control, (6) "Problem AZ only instant traffic exclude" → Zonal Shift, (7) "DR region prep state continuous validation" → Readiness Check. Next day digs into resilience's data layer (RDS, Aurora, DynamoDB Global Multi-AZ, Multi-Region choices).
Click a choice to reveal the answer and explanation.
Question 1
A team wants to be automatically assessed on whether a workload's current RTO/RPO meets its targets, receive concrete recommendations to close the gaps, and enforce resilience as a gate in the CI/CD pipeline. What is the most appropriate tool?
Question 2
A company wants to verify resilience in the production environment by deliberately injecting EC2 termination and network latency, with the experiment stopping automatically if the error rate exceeds a threshold. What is the most appropriate configuration?
Question 3
A company wants to keep operating its on-premises data center as is, maintain a DR environment on standby in AWS around the clock, and perform regular failover drills. What is the most appropriate service?
Question 4
A fintech operates active-active in us-east-1 and us-west-2. It wants to prevent traffic from swinging unnecessarily when automatic health checks misjudge a transient network blip as a failure, and wants operators to decide on failover carefully themselves during a failure. What is the most appropriate choice?
Question 5
In a chaos engineering experiment, what is the most accurate role of a Stop Condition?
Question 6
A workload is experiencing a partial failure in a single AZ. Without failing over the entire Region, you want to immediately remove only that problematic AZ from traffic. What is the most appropriate method?