Day 5 - Week 4 Comprehensive Review: Turning S3, CloudFront, and Storage Patterns into a Single Decision System

Week 4 covered the upper layers of the AWS storage stack. S3's internal consistency model and large-scale operational patterns (Day 16), the eight storage classes and the economics of lifecycle policies (Day 17), the five access-control layers and encryption key management (Day 18), and CloudFront's three-layer cache structure and hybrid storage patterns (Day 19). These four topics are the core of the "data storage" domain that appears most often on the exam.

Today's goal is to organize this not as rote memorization but as a decision system. Once the thought flow of "scenario keyword → design pattern → concrete service choice" is automatic, the answer becomes visible even in a scenario you've never seen.

A Connection Map of Week 4's Core Concepts

The four topics are not isolated islands. They sit on a single flow of data being created, consumed, protected, and expired.

[ Data flow and Week 4 service mapping ]

On-premises data              Global users
      |                             |
Storage Gateway ←────────────── CloudFront (CDN)
(File/Volume/Tape)               PoP → REC → Origin Shield
      |                             |
      └───────────→ S3 ←───────────┘
                   |  object storage core
                   |
          ┌────────┴────────┐
        Security           Cost
    BPA + Bucket         Lifecycle Policy
    Policy + KMS         (Hot→Warm→Cold→Ice)
    OAC + VPC EP         Intelligent-Tiering

Data flows around S3 at the center. On the left, on-premises Storage Gateway plays the hybrid-bridge role; on the right, CloudFront handles global distribution. Security and cost are the two core concerns wrapping around S3.

💡 Related theory: This architecture connects to the Data Mesh principle "treat data as a product (Data as a Product)." Keeping S3 as the central data lake, you provide a different interface per access pattern (on-premises connection, CDN distribution, direct API) while maintaining a single data source of record. This is thinking similar to CQRS (Command Query Responsibility Segregation), which supports diverse consumption patterns while maintaining a Single Source of Truth in a distributed system.

Storage Classes: A Summary of the Data-Temperature vs. Cost Trade-off

On the exam, storage class questions are almost always trade-off judgments. Approach them with the following framework and the answer comes in 2-3 seconds.

[ Storage class decision tree ]

Access frequency → immediate access needed → regenerable?
  │                │                          │
  │             YES (immediate)             YES → One Zone-IA (if AZ failure tolerable)
  │             :                           NO  → Standard or Standard-IA
  │
  ├─ Often (daily/weekly) → Standard
  ├─ Occasionally (monthly) → Standard-IA (30-day min, 128KB min)
  ├─ Once a quarter, immediate need → Glacier Instant Retrieval (90-day min)
  ├─ 1-2 times a year, time to spare → Glacier Flexible (3-5 hours or 1-5 minutes)
  ├─ Less than once a year, 12 hours OK → Glacier Deep Archive (180-day min)
  └─ Pattern unknown → Intelligent-Tiering (objects 128KB and up)

Three cost traps: If you don't meet the minimum storage duration (Standard-IA: 30 days, Glacier Instant/Flexible: 90 days, Deep Archive: 180 days), you're billed for the remaining period. The minimum object size (Standard-IA/One Zone-IA: 128KB). Incomplete multipart uploads are billed like completed objects.

💡 Related theory: Storage class choice connects to the information-theory concept of entropy. The more predictable the access pattern (low entropy), the more a manual Lifecycle policy is optimized. The more unpredictable (high entropy), the more Intelligent-Tiering's automatic classification lowers the total TCO. Intelligent-Tiering's monitoring fee is the "information cost" of prediction uncertainty.

📚 Case study: Netflix manages its entire content catalog across storage class tiers. New and popular titles are in Standard, less-popular content over 6 months old is in Standard-IA, and original master files are in Glacier Deep Archive. Lifecycle policies transition automatically, but when content regains popularity (a season 2 release, an award, etc.), it's raised back to Standard with S3 Batch Operations. This single pattern saves tens of millions of dollars a year in storage cost.

S3 Security: Five-Layer Evaluation and Keyword Mapping

The crux of a security question is "at which layer, and how, do you block/allow."

💡 Related theory: S3 access control's five-layer structure follows the same layering principle as the OSI model. Each layer operates independently, and if a lower layer denies, the allow of an upper layer is nullified. What NIST SP 800-207 "Zero Trust Architecture" calls "treat every request as potentially hostile and authenticate/authorize on every request" is implemented by this multi-layered structure. BPA is the first checkpoint, SCP is the organization boundary, the VPC Endpoint Policy is the network boundary, and IAM/Bucket Policy is identity-based control.

🔍 Going deeper: In envelope encryption, how is SSE-KMS's guarantee that "the key never leaves KMS" achieved? KMS's HSM (Hardware Security Module) is hardware certified to FIPS 140-2 Level 3. The master key exists physically only inside this HSM. When you call KMS's GenerateDataKey API, it generates a DEK (data encryption key) inside the HSM and returns the plaintext DEK and the DEK encrypted with the master key. After that, the master key never leaves the HSM, and only the encrypted DEK is stored. This structure is the crux of making key theft nearly impossible.

⚠️ Pitfall: Don't confuse Object Lock Governance Mode and Compliance Mode. If the requirement is "no one can delete," it's Compliance Mode. In Governance Mode, an IAM user with the s3:BypassGovernanceRetention permission (including root) can remove the lock. On the exam, when "regulatory-auditor requirement," "legal retention," or "even root can't delete" appear, it's Compliance Mode.

CloudFront: Criteria for Cache Layers and Edge Computing

CloudFront questions come in two patterns. "At which layer to process it" (cache layers) and "which edge compute to use" (Functions vs Lambda@Edge).

Cache-Layer Optimization

[ Design to minimize origin load ]

Goal: minimize the number of Origin requests

Layer 1: cache-key optimization
  → remove unnecessary headers/cookies from the cache key
  → treat the same content as the same cache entry

Layer 2: TTL optimization
  → static assets: 7+ days (use versioned filenames)
  → dynamic API: TTL=0 (no caching)
  → images: 1 day to 7 days

Layer 3: enable Origin Shield
  → multi-region users + single-origin structure
  → all REC cache misses consolidate to Origin Shield

Invalidation (cache invalidation): forces a cache refresh when a file is updated. Wildcards like /images/* are allowed. The first 1,000 paths/month are free, then billed per path. When updates are frequent, a strategy of including a version/hash in the filename to create a new cache without Invalidation is more efficient.

📚 Case study: When Facebook's 2021 DNS outage took services down for 6 hours, sites using CloudFront as their CDN kept serving static content with a high cache hit ratio, regardless of the origin failure. It's a case where CloudFront's TTL-based cache acted as a buffer against origin failure. However, API paths with TTL=0 took the full brunt of the origin failure.

Criteria for Choosing Edge Computing

🔍 Going deeper: The reason CloudFront Functions "can't make external calls" is not only a network constraint but also the execution-time constraint (1ms). Since the RTT of an external HTTP call is at least tens of ms, it can't complete within the 1ms limit. Lambda@Edge can run up to 30 seconds, so external API calls are possible. But Lambda@Edge runs at Regional Edge Caches (13 locations), so it's geographically farther from the user than CloudFront Functions (400+ PoPs). This is the technical basis for the principle "light, fast logic → Functions; heavy logic → Lambda@Edge."

Choosing a Hybrid Storage Tool: The Decisive Differences of the Three Services

Why Tape Gateway exists: There are still hundreds of thousands of physical tape libraries operating around the world. Backup software like Veeam, Veritas NetBackup, and IBM Spectrum Protect has decades of history, and replacement and retraining costs are enormous. Tape Gateway makes this software believe "there is a tape library" while actually storing data in AWS S3/Glacier. It removes the management burden of physical tape (tape swapping, off-site storage, tape degradation) while letting software migration be deferred to later.

💡 Related theory: Storage Gateway's local cache layer works on the same principle as a computer's memory hierarchy (Cache → RAM → Disk). Frequently accessed data (Hot) is in the gateway's local disk cache, and less-accessed data is in AWS S3/Glacier. In environments where the 80-20 rule applies (80% of all access concentrates on 20% of the data), setting the local cache size to 20-30% of the total data yields a high cache hit ratio.

Five Integrated Architecture Patterns

Here are the five patterns that appear most often in exam scenarios, organized as architectures.

Pattern A: Global static web service (the most common pattern)

User (global)
  → CloudFront (HTTPS, Signed URL/Cookie, WAF)
      ├─ /static/* → S3 bucket (BPA ON, OAC, SSE-KMS)
      │               Lifecycle: 90 days→IA, 365 days→Glacier
      ├─ /api/*    → ALB → ECS/EC2 (dynamic processing)
      └─ /video/*  → CloudFront Signed Cookie (premium subscribers)

Pattern B: Long-term retention of regulated data

Data creation → S3 bucket (SSE-KMS, Object Lock Compliance 7 years)
             → Lifecycle: immediately to Glacier Deep Archive
             → Versioning + MFA Delete
             → S3 Access Log → another bucket (for auditing)
             → CloudTrail + KMS API → audit log

Pattern C: ML training data pipeline

On-premises server → DataSync → S3 (training data)
                                 ↓ Lifecycle: after training, Standard-IA
                                 ↓ (FSx for Lustre integration)
                              EC2 GPU cluster → S3 (checkpoints)
                                              → S3 Object Lambda (data augmentation)

Pattern D: On-premises hybrid file sharing

Headquarters (Windows AD environment)
  └─ Storage Gateway FSx File Gateway
       ├─ local cache (frequently used files, immediate response)
       └─ Amazon FSx for Windows File Server (AD integration)
            └─ Lifecycle: old files → S3 Intelligent-Tiering

Pattern E: Multi-team data lake

S3 data-lake bucket (SSE-KMS, BPA ON)
  ├─ Access Point: analytics-ap (VPC-analytics only)
  │   Policy: s3:GetObject on /analytics/*
  ├─ Access Point: ml-ap (VPC-ml only)
  │   Policy: s3:* on /ml-data/*
  └─ Access Point: data-eng-ap
      Policy: s3:* on /* (full access for data engineering)

Bucket policy: Deny direct access, allow only Access Points
VPC Endpoint: each VPC → S3 (bypasses the internet)

⚠️ A roundup of 7 pitfalls

1. Reverting from Glacier to Standard via Lifecycle is impossible — only via CopyObject
2. S3 versioning can't be disabled, only Suspended — re-enabling recovers existing versions
3. OAI doesn't support SSE-KMS buckets — migration to OAC is needed
4. Lambda@Edge functions are created only in us-east-1 — CloudFront handles global distribution
5. CloudFront Functions handle only Viewer Request/Response — Origin events are Lambda@Edge
6. Storage Gateway Tape Gateway: for replacing physical tape — keeps on-premises backup SW
7. Transfer Acceleration: no effect or even slower within the same region — valid only across continents

A Comprehensive Cost-Optimization Checklist

For the SAA exam's cost-optimization domain (20% weight), approach storage-related questions with this checklist.

[ S3 cost-optimization checklist ]

□ Consider gp2 → gp3 migration (Week 3 content, includes EBS too)
□ Clean up S3 incomplete multipart uploads (Lifecycle: AbortIncompleteMultipartUpload 7 days)
□ Old-version expiration policy for versioned buckets (NoncurrentVersionExpiration)
□ Glacier transition for data unaccessed 90+ days (Lifecycle Transition)
□ Apply Intelligent-Tiering (unpredictable pattern + objects 128KB and up)
□ Identify unused data with S3 Inventory + S3 Analytics
□ Understand organization-wide storage patterns with Storage Lens
□ Raise the CloudFront cache hit ratio to reduce origin GET requests
□ Enable SSE-KMS → Bucket Keys (99% fewer KMS costs)
□ Transfer Acceleration: disable if unnecessary

📚 Case study: Until Dropbox migrated from AWS to its own data centers in 2016 (the "Magic Pocket" project), it stored the files of hundreds of millions of users on S3. During this period, the S3 cost-optimization patterns Dropbox applied were access-frequency-based storage class classification, multipart-upload optimization, and CloudFront integration to cut GET request costs. Its S3 cost at the time was on the order of tens of millions of dollars a year, and it disclosed that the optimization work saved 30-40%. This experience became the starting point for the industry discussion of "how to optimize exabyte-scale storage."

A Table of Key Numbers to Memorize

Here are the numbers that appear often on the exam, gathered in one place.

📝 Practice Questions

Click a choice to reveal the answer and explanation.

Question 1

A global media company streams 4K master videos stored in S3 to users worldwide. Which of the following is the correct design combination?

• AS3 Public bucket + CloudFront TTL=0 + no Lambda@Edge
• BS3 Private bucket + BPA enabled + CloudFront OAC + Signed Cookie + Origin Shield
• CS3 Public bucket + CloudFront OAI + Signed URL (per file)
• DS3 Private bucket + Transfer Acceleration + Signed URL

Question 2

A hospital stores MRI imaging data in S3. It's accessed frequently for 6 months after a visit, then needs immediate access 1-2 times a year for the next 5 years, and is deleted after 7 years. What is the optimal lifecycle policy?

• AGlacier Deep Archive immediately → delete after 7 years
• BDay 0: Standard → Day 180: Glacier Instant Retrieval → 7 years (2,555 days): delete
• CDay 0: Standard → Day 30: Standard-IA → Day 180: Glacier Instant → 7 years: delete
• DIntelligent-Tiering immediately → delete after 7 years

Question 3

Under SEC regulations, financial transaction records must be retained for 7 years, non-modifiable and non-deletable. No one, including the AWS root account, must be able to delete the data. How do you configure it?

• AS3 Versioning + MFA Delete
• BS3 Object Lock Governance Mode, Retention Period 7 years
• CS3 Object Lock Compliance Mode, Retention Period 7 years
• DS3 Bucket Policy: Deny s3:DeleteObject for all principals

Question 4

A data-analytics team's EC2 processes sensitive customer data in an S3 data lake. The security team requires two things: (1) prevent S3 data exfiltration over the internet, and (2) prevent the analytics EC2 from accessing any S3 outside the approved data-lake bucket. How do you configure it?

• AA policy on the EC2 IAM role allowing only a specific bucket + NAT Gateway
• BVPC Gateway Endpoint for S3 + Endpoint policy (approved buckets only) + S3 bucket policy (allow only from that VPC Endpoint)
• CEnable BPA on the S3 bucket + CloudFront OAC
• DAWS PrivateLink for S3 + Security Group restriction

Question 5

Production data at an on-premises factory is generated by a SCADA system. You want to store this data in AWS S3 in real time, but the SCADA system supports only the NFS protocol. The factory network and AWS are connected via Direct Connect. What is the most suitable solution?

• AInstall an AWS DataSync agent
• BAWS Storage Gateway S3 File Gateway
• CAmazon S3 Transfer Acceleration
• DAWS Snow Family

Question 6

An e-commerce company distributes product images worldwide via CloudFront. The image URLs carry ?size=medium&format=webp query parameters, but the parameter order differs per client (?format=webp&size=medium), so the same image is stored as a different cache entry. What is the most cost-effective way to raise the cache hit ratio?

• ASort the query parameters with Lambda@Edge (Origin Request)
• BSort the query parameters with CloudFront Functions (Viewer Request)
• CNormalize the query parameters at the ALB
• DExclude the query parameters from the cache key in the Cache Policy

Question 7

A startup stores user files in S3. Data is small initially but is expected to grow to hundreds of TB in 3 years. It's hard to predict each file's access frequency, and the team is small with no capacity for manual lifecycle management. What is the most suitable storage strategy?

• AStore all files in Standard and add Lifecycle later
• BStore all files in the Intelligent-Tiering storage class + a Lifecycle to clean up incomplete multipart
• CStore directly in Standard-IA
• DStore in Glacier Instant Retrieval

Question 8

A company encrypts data in an S3 bucket with a KMS key (SSE-KMS). A data-analytics system reads 5,000 S3 objects per second, and KMS API call costs are 5x higher than expected. How do you reduce cost while keeping encryption?

• AChange the encryption method to SSE-S3
• BEnable S3 Bucket Keys
• CDecrypt the data and store it in a cache
• DCreate the KMS key in a different region

Question 9

Which mapping of scenarios to solutions is correct? ① On-prem Veritas NetBackup writes to a tape library. Physical tape must go, but the backup software cannot be replaced. ② One-time migration of 10TB from on-prem to S3, with a 1Gbps Direct Connect link available. ③ Branch staff must keep accessing on-prem NAS data the way they always have. ④ Move 500TB collected at a remote construction site with no internet connectivity to AWS.

• A①-Tape Gateway, ②-DataSync, ③-S3 File Gateway, ④-Snowball Edge
• B①-DataSync, ②-Tape Gateway, ③-S3 File Gateway, ④-Snowball Edge
• C①-Tape Gateway, ②-Snowball Edge, ③-S3 File Gateway, ④-DataSync
• D①-S3 File Gateway, ②-DataSync, ③-Tape Gateway, ④-Snowball Edge

Question 10

Find the error in the following CloudFront Distribution configuration. A team tried to protect an S3 bucket with OAC and make it accessible only through CloudFront. Configuration: - S3 bucket: BPA disabled - CloudFront origin: uses the S3 static website endpoint (bucket.s3-website-ap-northeast-2.amazonaws.com) - OAC: configured - Bucket policy: allows the CloudFront OAC Principal

• ADisabling BPA is the problem
• BUsing the S3 static website endpoint as the origin is the problem
• COAC works only with SSE-S3
• DThe bucket policy is wrong

Question 11

A company wants to apply KMS encryption to all data in an S3 bucket and block uploads of unencrypted objects. How do you configure it?

• ASetting the S3 bucket default encryption to SSE-KMS automatically encrypts all objects
• BS3 bucket default encryption SSE-KMS + a bucket policy that Denies requests without the s3:x-amz-server-side-encryption-aws:kms header
• CReject requests without encryption in the KMS key policy
• DBlock unencrypted S3 PutObject with an SCP

Question 12

In an AWS Organizations environment, you want to centrally analyze the S3 data of 100 accounts. You need to see each account's bucket access patterns, cost-optimization opportunities, and security vulnerabilities at a glance. Which service is most suitable?

• AAWS Config + S3 rules
• BAmazon Macie
• CS3 Storage Lens
• DAWS Cost Explorer + S3 tags