Day 5 - Week 8 Comprehensive — 12 Serverless & Event Architecture Scenarios

Week 8 examined how serverless and event-driven architecture collaborate. Lambda (cold start, concurrency, VPC ENI), Step Functions (state machine, Saga, Distributed Map), EventBridge (Bus, Pipes, Scheduler), AppSync (GraphQL + Subscription), SQS/SNS/Kinesis/MSK (messaging, streaming) — six domains appear independent but production nearly always combines two or three. That's exactly what SAP-C02 exams target. "Lambda alone isn't enough," "Step Functions alone isn't enough" — the ability to identify keyword combinations and judge which tool combination fits best matters.

Today consolidates week 8 into 12 comprehensive scenarios. Each scenario asks not single tools but complex patterns, and explanations clarify why answers are correct and alternatives wrong using SAP exam decision criteria. Re-reading these scenarios before the exam redraws the entire domain's decision tree in your mind.

Week 8 Core Summary

Lambda (Day 1)

• Firecracker micro VMs, warmed in Slot pool, cold start is 4 stages: download/runtime/init/JIT
• 3 concurrency types: Account limit (1,000) / Reserved (isolation+ceiling) / Provisioned (warm, cost)
• SnapStart (CRaC-based): Java/Python/.NET, free, watch init uniqueness
• Hyperplane ENI (2019~): VPC integration shares ENI, RDS Proxy solves connection surge
• Async retry 2x + Destinations (SNS/SQS/EB/Lambda)
• Function URL (bypass API GW), Container Image 10GB + lazy loading, Graviton2 ARM 20%↓
• Burst Concurrency limit (us-east-1=3,000) → Predictable spikes use PC + Auto Scaling

Step Functions (Day 2)

• Distributed persistent state machine, Saga = Catch + separate compensate branch
• Standard (1yr, per-transition) vs Express (5min, per-invocation+time) — split by frequency/duration
• 7 State types + Distributed Map (10K parallel + S3 ItemReader)
• Integration 3 patterns: Request-Response / .sync (wait for job) / .waitForTaskToken (external callback)
• Retry + Catch: exponential backoff + JitterStrategy (2023)
• Visual Workflow Studio + auto IAM generation

EventBridge (Day 3)

• 3 Bus types: Default (AWS) / Custom (domain) / Partner (SaaS)
• Rule = Pattern + max 5 Targets, JSON deep-field matching + API Destination external HTTPS
• Pipes: SQS/Kinesis/DDB Stream/MQ/MSK → Filter → Enrich → Target, code-free
• Scheduler: 1M+ schedules, time zone, flexible window, 200+ direct API calls
• Schema Registry + Archive/Replay — evolution management + time travel
• SNS vs EventBridge: simple fan-out, high throughput vs rich filtering, diverse targets

AppSync + Messaging (Day 4)

• AppSync GraphQL: schema-first + direct resolver (DDB without Lambda) + Subscription (WebSocket) + 5 auth modes
• SQS Standard (unlimited TPS) vs FIFO (300 TPS, order+dedup 5min) + High Throughput FIFO
• Visibility Timeout, Long Polling, DLQ + maxReceiveCount
• SNS Fan-out + Message Filtering + FIFO Topic (2020)
• Kinesis Data Streams: replay + multi-consumer, Shard + Partition Key, EFO
• Firehose (60–900s buffer + Lambda transform), MSK (Kafka portability)
• Lambda ESM: BatchSize, ReportBatchItemFailures partial batch

Decision Tree — Scenario Keywords → Tool Mapping

"Cold start + zero cost"                    → SnapStart (Java/Python/.NET)
"Cold start + predictable traffic"          → Provisioned Concurrency + Auto Scaling
"Lambda + DB connection too many"           → RDS Proxy
"One function impacting others"             → Reserved Concurrency
"Java/Python Lambda 20%↓ cost"              → Graviton2 ARM switch
"ML model 1GB+ Lambda"                      → Container Image (10GB limit)

"Long-term workflow + compensate"           → Step Functions Standard + Catch
"High frequency, short workflow"            → Step Functions Express
"Human approval / external callback"        → .waitForTaskToken
"Wait for ECS/EMR/Glue job complete"       → .sync
"1M S3 objects in parallel"                 → Distributed Map + Express child

"1M user schedules"                         → EventBridge Scheduler
"SaaS partner event receive"                → EventBridge Partner Bus
"Queue → workflow, code-free"               → EventBridge Pipes
"Event replay, reprocess"                   → EventBridge Archive + Replay
"Rich filter + diverse targets"             → EventBridge Custom Bus + Rule

"GraphQL + real-time + mobile + auth"       → AppSync + Cognito
"Order + dedup payment queue"               → SQS FIFO
"Simple fan-out + very high throughput"     → SNS Topic + SQS subscription
"Replay + multi-consumer"                   → Kinesis Data Streams + EFO
"Auto-save to S3/Redshift/OpenSearch"       → Kinesis Firehose
"Kafka standard + portability"              → MSK
"Isolate failed after 5 retries"            → DLQ + maxReceiveCount
"Batch partial failure retry only"          → Lambda ESM ReportBatchItemFailures

SAP Exam Common Traps Summary

1. Task Role vs Task Execution Role (ECS/Fargate) — App permissions in Task Role, infra (ECR Pull, Logs) in Execution Role
2. EventBridge DLQ vs Lambda Destinations — EB DLQ for invoke failure, Lambda business logic failure uses Lambda Destinations
3. SQS FIFO "exactly-once" — Precisely: idempotency within 5-min dedup window; application-level idempotency still recommended
4. Standard vs Express cost — Express isn't always cheaper. Rarely-run long workflows cost less with Standard
5. Reserved Concurrency — Both isolation AND ceiling
6. SNS doesn't persist messages — Retention needed? → SQS subscription or EventBridge Archive
7. Pipes vs Step Functions — 1:1 routing is Pipes, complex workflow is Step Functions, often Pipes→SF combo
8. SnapStart uniqueness trap — Random numbers, UUIDs, DB connection state created at init get copied to all restored instances

📝 Practice Questions

Click a choice to reveal the answer and explanation.

Question 1

Global OTT service receives 50,000 viewer events/sec: (1) instant real-time recommendation model (2) 7-day raw event S3 preservation for ML retraining (3) analytics hourly aggregation for BI dashboard. Three downstream must process simultaneously without latency impact. Best architecture?

• ASNS Topic + 3 SQS subscriptions, each queue 14-day retention + DLQ, consumer polling
• BKinesis Data Streams + Enhanced Fan-Out (3 consumers) + Firehose
• CEventBridge Bus + 3 Rules, each with input transformer + 7-day Archive/Replay
• DMSK + 3 Kafka Consumer Groups, EBS gp3 7-day retention.ms

Question 2

Java Spring Boot Lambda cold start averages 4 sec. 1M daily calls, 5% cold ratio. Ops must reduce to <1sec without extra cost. Init has DB connection pool + UUID-based instance ID. Best combo?

• AProvisioned Concurrency = 500 via Auto Scaling schedule constant warm
• BEnable SnapStart only; leave UUID and DB connection init code as-is
• CSnapStart + move UUID generation and DB connection to Crac.Resource.afterRestore
• DIncrease memory to 10GB for CPU-proportional JIT/class loading speedup

Question 3

Payment → inventory deduct → shipping schedule workflow averages 30sec to days (external carrier response wait). Shipping failure requires payment refund + inventory restore. External carrier responds async via webhook. Best architecture?

• AStep Functions Express + Catch (compensate) + API GW polling for carrier webhook
• BStep Functions Standard + Catch (compensate) + .waitForTaskToken (webhook)
• CLambda chain + try/catch + DynamoDB state per step + call refund/restore compensate
• DEventBridge Pipes + Lambda enrichment routing payment→inventory→shipping sequentially

Question 4

1M-user SaaS sends daily summary emails at each user's preferred time (timezone + time). User setting changes apply immediately. Best architecture?

• A1M EventBridge Rules with per-user cron expressions + SES target
• B1M EventBridge Scheduler schedules + SES target
• CLambda + DynamoDB schedule table + 1-min cron poller scanning send targets
• D1M Step Functions Wait State workflows, each delay by user timezone offset

Question 5

Filter DynamoDB Streams events (change events), routing only matching patterns to Step Functions workflow. Want to avoid Lambda code and operations. Best architecture?

• ADDB Streams → Lambda (pattern filter code) → Step Functions StartExecution
• BDDB Streams → EventBridge Pipes (Filter) → Step Functions
• CDDB Streams → Kinesis Data Streams → Lambda consumer → Step Functions
• DDDB Streams → EventBridge Pipes → SQS → Lambda poller → Step Functions

Question 6

E-commerce fans out OrderPlaced event to 5 downstream (analytics, shipping, CRM, alerts, audit). Throughput 5,000/sec. Each downstream processes at own pace; failed messages permanently isolated for analysis. Best architecture?

• AEventBridge Custom Bus + 5 Rules (same pattern match) + DLQ per target
• BSNS Topic + 5 SQS subscriptions + each SQS's DLQ
• CKinesis Data Streams + 5 Consumers (EFO) + retry on consumer failure
• DLambda fan-out function sync-calls 5 downstream, failures to SQS

Question 7

Lambda function connects to RDS PostgreSQL inside VPC. Concurrent executions spike to 1,500, getting "FATAL: too many connections". RDS max_connections=200. Best fix?

• AUpgrade RDS instance class to max_connections=2,000 + Multi-AZ read distribution
• BLambda Reserved Concurrency = 200 to cap DB connections ≤ max_connections
• CIntroduce RDS Proxy; Lambda connects to Proxy
• DRemove Lambda VPC integration, expose PostgreSQL on public endpoint

Question 8

Mobile chat app uses GraphQL backend, receives message real-time push. Cognito user auth, DynamoDB storage, offline sync needed. Best backend?

• AAPI Gateway REST + Lambda + WebSocket API + DynamoDB message/connection ID management
• BAppSync GraphQL + Cognito User Pool + DynamoDB direct resolver + Amplify DataStore
• CALB + ECS Fargate Service + Socket.io + Cognito JWT verify middleware
• DIoT Core MQTT + device shadow for offline message sync

Question 9

1M CSV files in S3, each processed by Lambda. Per-file time 5–30 sec. Results aggregate to different S3 bucket. Best pattern?

• AMap state (inline, concurrent 40) iterates S3 object list, invokes Lambda
• BStep Functions Distributed Map + Express child + ItemReader/ResultWriter
• CAWS Batch submit 1M tasks + Fargate Spot + array job parallel
• DOrchestrator Lambda chunks 1M files, recursively calls child Lambdas

Question 10

Black Friday sale starts; Lambda concurrent execution spikes 0→5,000 in 5 minutes. us-east-1 burst limit 3,000. Must handle without throttle; exact start time known. Best architecture?

• AReserved Concurrency = 5,000 to isolate concurrency from other functions
• BApplication Auto Scaling + Provisioned Concurrency to 5,000 30 min before sale
• CIncrease function memory to 10GB to shrink needed concurrency via higher per-instance throughput
• DSwitch to Function URL to bypass API Gateway overhead, reduce latency

Question 11

Global SaaS receives Stripe webhook + Shopify webhook simultaneously, routes amount > $1,000 to Step Functions workflow, rest to Lambda. Avoid webhook receiver operations. Best architecture?

• AAPI Gateway + Lambda webhook receiver validates signatures, retries, branches by amount
• BEventBridge Partner Bus (Stripe + Shopify) + Rule (amount > 1000) + SF/Lambda targets
• CSNS HTTPS subscription receives webhook, message filter policy branches by amount
• DECS Fargate webhook proxy validates signatures, branches to SQS

Question 12

Fintech receives payment transactions in SQS, Lambda processes. Batch size=10; one failure triggers full batch retry (default), but idempotent-broken downstream causes duplicates. Retry only failed messages?

• ASet BatchSize=1
• BShorten Visibility Timeout
• CEnable ReportBatchItemFailures in Lambda ESM + function returns failed message IDs
• DSend to DLQ immediately

📌 Week 8 At a Glance

[Client]
   │ GraphQL
   ▼
[AppSync] ──Subscription(WebSocket)── Real-time mobile/web
   │ Mutation, direct resolver
   ▼
[DynamoDB] ──Streams──► [EventBridge Pipes]
                          │ Filter
                          ▼
                  [EventBridge Custom Bus]
                          │ Rule
        ┌─────────────────┼─────────────────┐
        ▼                 ▼                 ▼
  [Step Functions]    [SNS Topic]     [Kinesis Data Streams]
   ├─ Standard (1yr)   │ Fan-out       │ Retention 24h~365d
   ├─ Express (5min)   │ Filtering     │ EFO multi-consumer
   ├─ Saga(Catch)      └─ 5 SQS        │
   ├─ Distributed Map                   │
   ├─ .sync (job wait)                  ▼
   └─ .waitForTaskToken (external callback) [Firehose] ──► S3/Redshift

[Lambda]
 ├─ Concurrency 3 types (account/Reserved/Provisioned)
 ├─ SnapStart (Java/Python/.NET, free)
 ├─ Hyperplane ENI (VPC), RDS Proxy connection
 ├─ Destinations (OnSuccess/OnFailure → SNS/SQS/EB/Lambda)
 ├─ Container Image 10GB (block-level lazy loading)
 └─ Graviton2 ARM 20%↓

[EventBridge Scheduler] ──► 1M+ user schedules
[EventBridge Partner Bus] ──► Stripe/Shopify/Datadog direct
[EventBridge Archive + Replay] ──► Event reprocessing
[Schema Registry] ──► Evolution management + code binding

Pre-Exam Checklist

• Can explain Lambda cold start 4 stages (download/runtime/init/JIT)
• Understand SnapStart mechanics (CRaC snapshot + restore) and uniqueness trap
• Explain Reserved vs Provisioned vs account limit in one sentence
• Know how RDS Proxy solves connection surge
• Standard vs Express pricing model and split criteria
• When to use .sync vs .waitForTaskToken
• Distributed Map ItemReader/ResultWriter + Express child
• EventBridge 3 Bus types + Rule + 5 Target limit
• Pipes 4 stages: Source/Filter/Enrich/Target
• Scheduler vs CloudWatch Rule cron limits
• SNS vs EventBridge decision (simple fan-out vs rich filter)
• SQS Standard vs FIFO + High Throughput FIFO
• Visibility Timeout, Long Polling, DLQ + maxReceiveCount
• Kinesis Shard + Partition Key + EFO
• Lambda ESM ReportBatchItemFailures partial batch

Next week (Week 9): Data Architecture — Data Lake (S3 + Lake Formation + Glue), Redshift (RA3 + AQUA + Spectrum), EMR (Spark/Hive on EC2/EKS), Athena (Iceberg + federated query), MSK Connect, Lakeformation permission model, Migration to Data Lakehouse.