Day 5 - Week 6 Comprehensive Review: The Complete DynamoDB Map

This is the last day of Week 6. DynamoDB is the service that generates more exam questions than any other on DVA-C02, and plain memorization won't hold up against reworded variations. Today we connect everything the week covered — the mathematics of partition key design, the read consistency models, the internal differences between LSI and GSI, the DAX cache layer, Streams plus Lambda integration, the two-phase commit behind transactions, and TTL's asynchronous deletion — into a single map, then close with comprehensive questions in real exam format. Someone who looks at DynamoDB through the lens of "how is data actually stored and read at the partition level" and someone who has only memorized API names will get completely different results on the same question.

The Complete DynamoDB Architecture Map

Client request
      │
      ▼
  [DAX cluster]  ─── cache hit → returns in μs
      │ cache miss
      ▼
[DynamoDB request router]
      │
      ▼
[Partition metadata service]
  Partition key → hash → determines the owning storage node
      │
      ▼
[Storage node (B-Tree + WAL)]
  Primary Node ──replicate──► Replica 1
               ──replicate──► Replica 2 (spread across 3 AZs)
      │
      ├── Read (Eventually Consistent): any replica
      ├── Read (Strongly Consistent): Primary only
      └── Write: Primary → WAL → replication complete
      │
      ▼
[DynamoDB Streams shard]  (changes retained 24 hours)
      │
      ▼
[Lambda ESM polling]
  Batch processing → BisectBatchOnFunctionError → DLQ

💡 Related theory: DynamoDB's internal storage is a B-Tree-based LSM (Log-Structured Merge-tree). On a write, the change is first recorded in the Write-Ahead Log (WAL); once the Primary node acknowledges the write, it propagates to the remaining replicas in the background. An Eventually Consistent read can hit a replica before that propagation finishes, which is how you end up seeing stale data. A Strongly Consistent read goes only to the Primary, so it is always current — but it consumes twice the RCU.

Partition Key Design — The Mathematics of Hot Partitions

Per-partition limits: 3,000 RCU + 1,000 WCU + 10GB

When items concentrate on one partition, that partition hits its limit and throws ProvisionedThroughputExceededException — the hot partition problem.

Bad partition key examples:
  status = "PENDING" → most new orders land on a single partition
  date = "2026-06-27" → all of today's data on a single partition

Good partition key examples:
  orderId = UUID → random distribution
  userId + timestamp → distributed per user, sorted by time

Write sharding pattern:
  partition_key = userId + "#" + str(random.randint(0, 9))
  → the same userId is spread across 10 partitions
  → reads must Query all of 0~9 and merge the results

🔍 Going deeper: Adaptive Capacity, introduced in 2018, automatically shifts spare capacity from other partitions when traffic to one partition rises, so a hot partition can temporarily survive exceeding its provisioned share. Burst Capacity separately accumulates whatever capacity a partition left unused over the last 300 seconds (5 minutes), banking up to 300 seconds' worth as reserve and spending it on a sudden spike. Both operate automatically, but neither is a real fix for a sustained hot partition.

RCU/WCU Mathematics: A Quick Reference

Reads (unit: 4KB, rounded up):
  Eventually Consistent = ceil(size/4KB) × 0.5
  Strongly Consistent   = ceil(size/4KB) × 1
  Transactional Read    = ceil(size/4KB) × 2

Writes (unit: 1KB, rounded up):
  Standard Write        = ceil(size/1KB) × 1
  Transactional Write   = ceil(size/1KB) × 2

Key point: reads are measured in 4KB, writes in 1KB → for the same item you need 4× more WCU than RCU

Worked examples:

6.5KB item, strongly consistent, 40 per second:
  ceil(6.5/4) = 2 → 2 × 1 × 40 = 80 RCU

3KB item, transactional write, 20 per second:
  ceil(3/1) = 3 → 3 × 2 × 20 = 120 WCU

⚠️ Trap: FilterExpression does not reduce RCU. Items that don't match the condition are still read and then filtered out, so the RCU is already spent. To save RCU you have to reduce the number of items read in the first place with KeyConditionExpression. The correct approach is to create a GSI whose partition key matches your query pattern.

LSI vs GSI: How They Work Internally

💡 Related theory: An LSI shares the base table's partition key, so its data is stored together in the same partition. That co-location is what lets the base table's Primary node manage the LSI data as well, which in turn makes Strongly Consistent reads possible. A GSI is replicated asynchronously into a completely separate partition space, so replication lag exists and only Eventually Consistent reads are supported. If you think of a GSI as "a separate table," this difference feels natural.

The Sparse Index pattern: if you use an attribute that not every item carries as a GSI's partition key, only the items that have that attribute show up in the GSI.

Users table:
  userId(PK) | email | isPremium (present on some items only)

GSI: isPremium-index
  PK = isPremium → only items carrying the isPremium attribute are indexed
  Query "isPremium = true" → efficiently retrieves premium users only
  Regular users without isPremium are absent from the GSI entirely → storage cost saved

📚 Case study: When Airbnb adopted DynamoDB Single-Table Design, it used a GSI sparse index for reservation status management. Only items whose reservation status was "PENDING" were given a pendingAt attribute, which served as the GSI's partition key. When a reservation is confirmed, pendingAt is deleted and the item drops out of the GSI automatically. That makes "all currently PENDING reservations" a GSI Query costing O(number of PENDING items), with no full table Scan.

The DAX Cache Layer — When Does It Get Bypassed?

DAX caches:
  GetItem → Item Cache (default TTL 5 minutes)
  Query   → Query Cache (default TTL 5 minutes)
  Scan    → Query Cache

DAX is bypassed (goes straight to DynamoDB):
  ConsistentRead=true strongly consistent reads
  TransactGetItems
  All writes (Put/Update/Delete)

DAX writes are write-through: write to DynamoDB first → update the Item Cache → return completion. It isn't write-back, so there is no risk of data loss.

⚠️ Trap: DAX is reachable only from inside a VPC. There is no direct access from the internet. For a Lambda function to use DAX, that Lambda must sit in the same VPC, and the DAX cluster's security group must allow the Lambda's security group. A DAX cluster needs at least 3 nodes (Multi-AZ) for high availability.

DynamoDB Streams + Lambda ESM in Detail

Stream view types:
  KEYS_ONLY:          PK + SK only (cheapest)
  NEW_IMAGE:          the whole item after the change
  OLD_IMAGE:          the whole item before the change
  NEW_AND_OLD_IMAGES: both before and after (audit logging)

Key Lambda ESM settings:
  BatchSize:                    1~10,000 records
  BisectBatchOnFunctionError:   on error, split the batch in half and retry
  MaximumRetryAttempts:         maximum retry count (default: unlimited)
  DestinationConfig:            failed records → SQS DLQ or SNS
  ParallelizationFactor:        parallel Lambdas per shard (1~10)
  FilterCriteria:               trigger Lambda only on matching event patterns

💡 Related theory: DynamoDB Streams runs on a Kinesis-based architecture internally. There is one stream shard per partition, and within a shard the ordering of events per partition key is guaranteed. Lambda ESM polls the shards — Streams does not "push" to Lambda; Lambda "pulls" from Streams. Setting ParallelizationFactor=10 lets 10 Lambdas process a single shard in parallel, but it complicates the ordering guarantee for events sharing a partition key.

A FilterCriteria example — process INSERT events only:

{
  "Filters": [
    {
      "Pattern": "{\"eventName\": [\"INSERT\"]}"
    }
  ]
}

Identifying a TTL deletion event inside the Lambda function:

def handler(event, context):
    for record in event['Records']:
        if record['eventName'] == 'REMOVE':
            user_identity = record.get('userIdentity', {})
            if user_identity.get('type') == 'Service':
                # Automatic deletion by TTL → skip the business logic
                continue
            # Explicit DeleteItem call → record an audit log
            log_deletion(record['dynamodb']['OldImage'])

📚 Case study: A 2019 DoorDash case. In a pipeline that received order-completion events from DynamoDB Streams and updated an ElasticSearch index, the Lambda was configured with BatchSize=100. When orders surged, processing fell behind, the Lambda timed out, the entire batch failed, and the same 100 records fell into a loop of endless retries. The problem was that neither BisectBatchOnFunctionError=true nor DestinationConfig (a DLQ) had been configured. After the fix, the pipeline isolated poison-pill messages to a DLQ while normal records kept flowing.

Transaction vs Conditional Write: How to Choose

Single item, condition needed → ConditionExpression (fast and cheap)
Multiple items need atomicity → TransactWriteItems (2× cost)
Reads and writes need atomicity → TransactGetItems + TransactWriteItems
Idempotent API request → ClientRequestToken (valid 10 minutes)

PITR vs On-Demand Backup vs TTL

🔍 Going deeper: PITR is capped at 35 days because AWS internally retains the change log (WAL) for 35 days. If you additionally connect Kinesis Data Streams for DynamoDB, you can retain the change history for up to 365 days, and by building a Kinesis Firehose → S3 pipeline you can keep it for years under an S3 lifecycle policy. When you need long-term audit logs, Streams + Kinesis is the standard answer.

Global Tables — The Implications of Active-Active Design

Conflict resolution: Last Writer Wins (latest timestamp wins)
Replication:         asynchronous, based on DynamoDB Streams
Consistency:         local strong consistency within each region only
Requirements:        Streams must be enabled; on-demand or identical provisioning

The trap you need to know before using Global Tables: "global Strongly Consistent" does not exist. If data written in Seoul hasn't yet replicated to Virginia, a strongly consistent read in Virginia still won't see Seoul's latest data. Each region guarantees consistency only against its own local replica.

⚠️ Trap: When two regions modify the same item simultaneously in Global Tables, the conflict is resolved by Last Writer Wins. Both regions return a success response, yet one of the writes ends up overwritten. For something like financial data, where both writes must be durable, Global Tables' conflict resolution policy may simply be unsuitable. In that case you should choose an architecture that permits writes only in a single "primary" region and keeps the other regions read-only.

Core Numbers to Memorize

Max item size:               400KB
Transaction max:             100 items, 4MB
BatchWriteItem:              25 items, 16MB
BatchGetItem:                100 items, 16MB
Streams retention:           24 hours (fixed)
Kinesis for DDB retention:   up to 365 days
PITR restore window:         35 days
TTL deletion delay:          0~48 hours
On-demand mode switch:       once per 24 hours
Partition limits:            3,000 RCU + 1,000 WCU + 10GB
GSI default max:             20
LSI max:                     5 (at creation only)
DAX default TTL:             5 minutes (Item Cache, Query Cache)
Transaction ClientRequestToken: valid 10 minutes