Based on the DOP-C02 exam guideReport an error
Week 13 addressed "how do you design systems that survive failures, and how do you validate that survival actually works?" There is one single insight threading through this entire week — availability isn't "preventing failures from occurring" but "assuming failure will occur, designing tolerance through replication, quorum, and failover, and validating that tolerance through experiments." From single AZ to multi-AZ, from single region to multi-region, from backup to Active-Active, and from "it should work" belief to FIS chaos experiments — the week's flow progressively assumes stronger failures and validates more rigorously.
Today we re-weave the four days' core concepts and comprehensively check understanding through scenario problems of the type actually seen on the DOP exam. The exam's pitfalls almost always lie in "subtle trade-off differences between similar-looking choices" — synchronous vs. asynchronous, single reader vs. multi-master, keeping infrastructure on vs. off by default, Compliance Mode vs. Governance Mode. We clarify these boundaries again.
Replication is the RPO-latency trade-off between synchronous and asynchronous, positioned on CAP/PACELC coordinates. RDS Multi-AZ uses block-level physical synchronous replication so Standby acts like a shadow disk—unable to serve reads—while Read Replica uses logical asynchronous replication able to serve reads from its own query engine — this physical/logical difference explains "why one can't read and the other can." Aurora internalizes replication to storage via 6-copy/3-AZ quorum (4/6 writes, 3/6 reads, AZ+1 fault model) and "log is database" design, ahead of traditional DBs. RTO depends on "recovery candidates being hot and data copying not needed," and RDS Proxy hides failover from clients through connection pooling.
Multi-region must accept asynchronous replication due to speed of light, and handling that conflicts/inconsistency is the essence. Route 53 is DNS-based GSLB (7 routing policies + quorum health checks) but limited by DNS cache TTL delays, while Global Accelerator with anycast fixed IPs handles immediate failover. Data replication is a single reader (Aurora Global, no conflicts) vs. multi-master (DynamoDB Global Tables, LWW) trade-off, and LWW carries silent lost update risks from clock skew. KMS Multi-Region Keys share the same key material across regions giving replicated ciphertexts portability, and Route 53 ARC provides safe DR toggling through control/data plane separation.
DR is not "how fast" but using BIA to decide RTO/RPO and balance against cost (ISO 22301), an economic problem. The 4 strategies are a spectrum of "how much to keep on by default" — Backup & Restore (nothing on, cheapest and slowest but IaC shortens RTO) → Pilot Light (data only, app off, path unvalidated) → Warm Standby (scaled-down instance, path validated) → Active-Active (full capacity both, RTO 0 but expensive and needs full-capacity planning for "half down, all throughput"). AWS Backup unifies policy with tags, and Vault Lock Compliance Mode makes immutable backups with WORM to stop ransomware.
Untested recovery doesn't work. Chaos Engineering (originating Netflix Chaos Monkey) applies scientific method (steady state → hypothesis → failure injection → validation) to reliability. FIS implements chaos management with Targets (blast radius dial: ALL/COUNT/PERCENT), Actions, and Stop Conditions (safety belt), while Resilience Hub quantifies RTO/RPO goal gaps. EventBridge Scheduler + FIS automates periodic iteration, and Game Day (people/one-time) and Chaos (system/periodic) complement each other.
| Confusing Pair | Decisive Difference |
|---|---|
| RDS Multi-AZ Standby vs Read Replica | Physical sync (can't read) vs logical async (can read) |
| Aurora vs Aurora Global | Single region 6-copy(30s) vs multi-region async(promote in minutes) |
| Aurora Global vs DDB Global Tables | Single reader (no conflicts) vs multi-master (LWW) |
| Route 53 Failover vs Global Accelerator | DNS (cache TTL delay) vs anycast IP (immediate) |
| Pilot Light vs Warm Standby | App off (unvalidated) vs scaled-down instance (validated) |
| Vault Lock Governance vs Compliance | Authority can override vs root cannot (WORM) |
| Backtrack vs PITR | In-place rewind (MySQL, 72h) vs restore to new cluster |
| Game Day vs Chaos | People/one-time vs system/periodic automated |
Click a choice to reveal the answer and explanation.
Question 1
A global gaming service uses UDP traffic and must failover instantly to another region if one dies—without client DNS cache delays. Simultaneously, clients should know only 2 fixed IPs. Most appropriate?
Question 2
Payment system needs RTO 0, RPO 0 or near it, cost almost unconstrained. Must accept simultaneous writes across two regions. Data layer configuration and critical validation risks?
Question 3
Traditional RDS Multi-AZ system experienced growing read load. Attempted to offload reads to Standby but found it impossible. Reason and correct solution?
Question 4
Aurora MySQL operator accidentally ran a large UPDATE. Need to rewind entire cluster to 5 minutes ago fastest. Separately, "specific table from 30 days ago"—what's the difference?
Question 5
Lambda serverless app fails with RDS "too many connections" on traffic spike, and DB failover disconnects all—connection storm erupts. Solve both in one go, standard approach?
Question 6
Finance regulation mandates backups remain unmodifiable within retention period—root user cannot delete or alter to stop ransomware erasing backups. Correct setup?
Question 7
Tier 3 non-critical workload, minimize DR cost. All infrastructure Terraform-coded, RTO allows hours. Most appropriate strategy and why?
Question 8
Automate Pilot Light DR activation into code. Correct stages and tools?
Question 9
30% random production EC2 instances, 5 min CPU load, validate Auto Scaling self-heal, halt+rollback if real disaster looms. FIS config?
Question 10
Frequent code/infra changes; resilience silently breaks (regression). Best operational pattern to prevent?
Question 11
Measure actual resilience against RTO/RPO goals (AZ failure RTO 600s) and auto-receive improvement proposals with cost impact. Best service?
Question 12
Global users served from lowest-latency region by default, one region dies → auto-serve only healthy regions, data async-replicated (relational DB, writes single location). Right combo?
This week's conclusion is singular — availability isn't preventing failure but designing tolerance assuming failure and validating that tolerance experimentally. Day 1 built single-region HA via replication/quorum (Aurora 6-copy/3-AZ, CAP/PACELC); Day 2 expanded to multi-region via async global replication (Aurora Global single reader vs DDB Global Tables multi-master LWW) plus DNS/anycast routing and KMS MRK encryption boundary; Day 3 set DR 4 strategies ("how much kept on by default" spectrum) via BIA RTO/RPO economics and Vault Lock WORM immutable backups; Day 4 solved "untested recovery fails" via Netflix-origin chaos engineering, FIS (blast radius + Stop Condition), and Resilience Hub (quantification). Exam success hinges on reading subtle boundaries: sync/async, single reader/multi-master, Pilot Light/Warm Standby, Governance/Compliance Mode, Game Day/Chaos.
Security Automation — GuardDuty, Security Hub, Config, Audit Manager
💪 Week 13 complete! You've internalized the mindset: don't fear failure, design failure in, validate rigorously.