Day 3 - ElastiCache: How an In-Memory Cache Saves Your Database

Caches exist at every layer of a computer system. CPU L1/L2/L3, the OS page cache, the disk controller cache, the browser cache, the CDN — and between the application and the DB sits an in-memory cache. ElastiCache is the managed service that fills that spot, offering two engines, Redis and Memcached, from the same console. Both are in-memory, but their design philosophies are opposite — Redis bills itself as a "data structure server," and Memcached as a "simple distributed memory pool." This difference is the point you need to distinguish both on the exam and in practice.

On DVA-C02, ElastiCache usually shows up in two forms. ① "DB load is too high + the same queries repeat" → introduce a cache. ② "Share sessions across multiple EC2/ECS instances" → ElastiCache Redis. And one level deeper, it asks "why is the cache showing stale data" — that is, the cache invalidation strategy. Today's material covers all three layers.

Why Caches Exist — An Application of Locality of Reference

The fundamental principle that makes an external cache like ElastiCache work is locality of reference, discovered by computer architecture in the 1960s. Temporal locality says "recently accessed data is likely to be accessed again soon," and spatial locality says "when you access one piece of data, nearby data is likely to be accessed too." In web services, objects like user profiles, product information, session data, and popular posts all have very high temporal locality — a single user views their own profile N times within 5 minutes.

A single disk I/O on a DB is 100μs ~ 1ms even on SSD, and 5-10ms on HDD. Fetching the same data from RAM takes under 100ns. That is a difference of roughly 10,000x to 100,000x. Picture an SNS feed where one page render needs 100 objects: the difference between making 100 round trips to the DB and 100 round trips to the cache splits page response time into something like 5 seconds vs. 5ms.

The standard formula for quantifying cache effect is Average Memory Access Time (AMAT) = Hit_time + Miss_rate × Miss_penalty. If Hit_time is 0.5ms, Miss_rate is 5%, and Miss_penalty is 50ms (DB lookup + cache store), then AMAT = 0.5 + 0.05 × 50 = 3ms. Lowering Miss_rate to 1% drops AMAT to 1ms. This is why average latency can drop sharply from even a 1-2% improvement in cache hit ratio. In a "latency didn't drop after introducing the cache" scenario, suspect the hit ratio.

Redis vs Memcached: A Difference in Design Philosophy

Memcached was created in 2003 by Brad Fitzpatrick of LiveJournal, and it does exactly one thing: "distributed memory cache." Its data model is simply key → byte string, with a single TTL option. It is written multithreaded, so throughput grows linearly with the number of cores on a node.

Redis was created in 2009 by Salvatore Sanfilippo and bills itself as a data structure server.

Beyond String, it provides 9 data structures in memory — List, Hash, Set, Sorted Set, Stream, HyperLogLog, Geo, Bitmap — and supports rich operations of O(log N) or better on each.

Because Redis is built on a single-threaded event loop (libevent-based), atomic operations come naturally, and in exchange you need multiple instances/shards to use more than one core.

The reason Redis is fast despite being single-threaded is that every command is an in-memory O(1) ~ O(log N) operation, and network I/O is handled asynchronously on libevent's epoll. A single core can handle 100,000 QPS because each command takes only microseconds, so context-switch cost would actually be a loss. From Redis 6.0, only network I/O is split off into a separate thread pool ("I/O threading") to use multiple cores to some extent, but command execution itself remains single-threaded. This design gets you both atomicity guarantees and performance.

The answer to the scenario "build a game leaderboard with ElastiCache" is the Redis Sorted Set. ZADD/ZRANGE/ZREVRANK run in O(log N), handling a leaderboard of hundreds of thousands of players in milliseconds. It's hard to implement the same functionality with Memcached (you'd have to load everything and sort it every time). When you see the words "leaderboard," "ranking," or "real-time score," choose Redis.

The 5 Caching Strategies — Five Ways to Use the Same Tool

Even on the same ElastiCache instance, system consistency and performance differ completely depending on the policy of what you fill and what you evict, and when. The exam asks you to clearly distinguish these 5 strategies.

Lazy Loading (Cache-Aside)

The most widely used pattern. The cache is filled only when a read request comes in. On a miss, it queries the DB and then stores the result in the cache.

def get_product(product_id):
    cached = cache.get(f'product:{product_id}')
    if cached:
        return json.loads(cached)          # cache hit
    item = db.get_item(Key={'id': product_id})['Item']
    cache.setex(f'product:{product_id}', 3600, json.dumps(item))  # 1h TTL
    return item

• Pro: only data that was actually queried is cached → memory efficient
• Con: the first request is always slow (cold miss). Also, the cache doesn't know when the DB is updated, so there's a risk of stale data

Write-Through

Updates the DB and the cache at the same time at write time. The cache is always up to date.

def update_product(product_id, data):
    db.put_item(Item=data)
    cache.setex(f'product:{product_id}', 3600, json.dumps(data))

• Pro: guarantees data freshness
• Con: data that will never be read also goes into the cache → wasted memory. Write latency increases

Write-Back (Write-Behind)

On write, only the cache is updated first, and the data is flushed to the DB asynchronously in the background. Maximizes write performance.

• Pro: very high write throughput
• Con: if the cache dies, unapplied data is lost. Data consistency is hard to guarantee, so rather than implementing it directly with ElastiCache, a pattern with a queue like Kafka/Kinesis in between is recommended
• It's important to know it as an anti-pattern

Read-Through

When the application makes a request to the cache library, the cache library automatically goes to the DB itself. The user code only calls cache.get.

• Pro: simple calling code
• Con: ElastiCache doesn't support native read-through — it is implemented at the framework level, such as Hibernate L2 cache or Spring Cache

Refresh-Ahead

Refreshes in the background just before the TTL expires so that users never see a cache miss.

• Pro: constant user latency
• Con: if the prediction is off, unnecessary DB calls. Applying it to every key makes the load spike

The reason cache invalidation is hard is that when a distributed system's source of truth is replicated in multiple places, propagating a change consistently to every replica is inherently a distributed consensus problem. In an ElastiCache + RDS combination, the trap that "data updated with a direct UPDATE on RDS is not refreshed in the cache" leads to real-world incidents. The solutions are ① force every write to go through the application ② publish cache invalidation events through CDC such as DDB Streams/Debezium ③ limit the stale window with a short TTL.

When a popular post's cache expires, thousands of concurrent requests rushing at the DB is called a thundering herd. Facebook introduced a lease token mechanism to handle this in its Memcached clusters (the "Scaling Memcache at Facebook" paper, NSDI 2013). A token is issued only to the first miss request, and the other requests wait briefly or use stale data. In Redis, this pattern is implemented with SET NX EX (the distributed lock pattern) or a single-flight pattern. Using Lazy Loading naively on its own causes a similar problem at the moment popular content expires.

Redis Cluster Mode: Distributing Data Across 16,384 Slots

ElastiCache Redis has two topologies.

Cluster Mode Disabled: a single shard with 1 primary + up to 5 replicas. All keys go to one node, so the memory limit equals the instance's memory size (e.g., cache.r7g.xlarge = 25GB).

Cluster Mode Enabled: multiple shards (up to 500) × each (1 primary + up to 5 replicas). Redis's own hash slot mechanism distributes keys across 16,384 slots and maps slots to shards. The key hashing function is CRC16(key) mod 16384.

Key "user:123"
   ↓ CRC16
Slot 5474
   ↓ mapping table
Shard 3
   ↓
Shard 3's primary node

16,384 is not an arbitrary number. According to design notes by Redis author antirez (Salvatore Sanfilippo), 16,384 is the balance point of the following three things.

• Every node must hold the slot mapping information, and 16K slots fit in a bitmap of ~2KB.
• Since the maximum number of Redis cluster nodes was designed at around 1,000, a slot-to-node ratio of 16:1 gives good evenness of data distribution.
• It also affects message size when cluster messages are exchanged over the gossip protocol.

Enabling Cluster Mode puts constraints on transactions and multi-key operations. MULTI/EXEC or MGET require all keys to be in the same slot. The solution is a hash tag, which forces only the part inside {...} in the key name to be used for hashing. Example: user:{123}:profile and user:{123}:settings both hash only the {123} part and so land in the same slot, so MGET user:{123}:profile user:{123}:settings is possible.

⚠️ Trap: When migrating to Cluster Mode Enabled, the client library must support the Cluster protocol. For Java, the Jedis Cluster client; for Python, redis-py-cluster (or RedisCluster in redis-py 4.x). Connecting to the cluster endpoint with an ordinary Redis client errors out because it can't handle MOVED/ASK redirects. On the exam, "after migrating to ElastiCache, some commands fail" → check the client's cluster support.

Redis Persistence: The RDB and AOF Trade-off

The reason Redis offers persistence options even though it is in-memory is to avoid a cache cold start on restart. If the cache is empty, all requests rush to the DB and cause a thundering herd incident. There are two persistence mechanisms.

RDB (Redis Database): dumps the entire memory to disk as a binary snapshot at set intervals (e.g., when 100 keys change within 5 minutes). Small file size and fast restore. Downside: data after the last snapshot is lost.

AOF (Append-Only File): appends every write command to disk as a text log. Data loss is nearly 0 (with the fsync everysec option). Downside: large file size, and restore requires replaying every command, so it is slow.

ElastiCache supports automatic backups (one RDB snapshot per day, stored in S3) and manual backups. AOF is disabled in ElastiCache Redis Cluster Mode Enabled, and Multi-AZ replicas are recommended instead.

The reasons ElastiCache doesn't recommend AOF are ① fsync cost reduces throughput by 30-50% ② the same durability can be obtained with Multi-AZ replicas ③ the service is unavailable during AOF replay time. In other words, for cache use the cost-benefit doesn't add up.

MemoryDB (launched in 2021) is a separate service that solves exactly this problem, providing strongly consistent + durable Redis with a multi-AZ transaction log (a mechanism similar to DynamoDB). The price is about 2-3x that of ElastiCache.

Session Management: ElastiCache's Killer Use Case

In a stateful web application with multiple EC2/ECS instances behind an ALB, if a user is routed to a different instance on every request, a session data sharing problem arises. In the past it was worked around with ALB sticky sessions, but this has the problems of ① session loss when an instance goes down ② uneven load ③ conflict with Auto Scaling.

If you externalize sessions to ElastiCache Redis, every instance sees the same cache. Implementation pattern:

import redis, secrets, json, time
 
cache = redis.Redis(host='session-cache.xxxx.cache.amazonaws.com',
                    port=6379, ssl=True, decode_responses=True)
 
SESSION_TTL = 1800  # 30 minutes
 
def create_session(user_id):
    sid = secrets.token_urlsafe(32)
    data = {'user_id': user_id, 'created_at': time.time(),
            'csrf_token': secrets.token_urlsafe(16)}
    cache.setex(f'sess:{sid}', SESSION_TTL, json.dumps(data))
    return sid
 
def get_session(sid):
    raw = cache.get(f'sess:{sid}')
    if raw is None:
        return None
    # sliding window: refresh the TTL on every access (keeps the active session)
    cache.expire(f'sess:{sid}', SESSION_TTL)
    return json.loads(raw)
 
def invalidate_session(sid):
    cache.delete(f'sess:{sid}')  # logout

If you manage sessions with ALB sticky sessions, when one node goes down, the sessions of all users tied to that node disappear. If you keep sessions in ElastiCache Redis and use Multi-AZ + automatic failover, sessions survive even when an instance goes down.

Security: AUTH, In-Transit Encryption, IAM Integration

ElastiCache security consists of three layers.

Network: only VPC-internal endpoints are provided (no public endpoint). Use Security Groups to allow port 6379 (Redis)/11211 (Memcached) only from the application SG.

Authentication: Redis uses an AUTH token (Redis 5+) or RBAC (Redis 6+, ElastiCache 6.0+). RBAC can grant per-user command/key ACLs (e.g., a read-only user is allowed only GET/HGET). Since 2023, IAM authentication for ElastiCache Redis has been added and works much like RDS IAM Auth — allow elasticache:Connect in the IAM policy + map IAM authentication to the Redis user.

Encryption:

• In-transit: TLS enable option (set at creation; changing it afterward requires backup/restore)
• At-rest: EBS volume encryption with KMS

⚠️ Trap: ElastiCache cannot have a public IP. To connect from outside, use one of three: ① VPN ② Direct Connect ③ EC2 bastion + SSH tunnel. The answer to the scenario "access ElastiCache directly from on-premises" is VPN/Direct Connect.

ElastiCache Serverless (GA in 2023)

Traditional ElastiCache requires you to choose a node class (cache.r7g.large, etc.). The Serverless option delegates capacity decisions to AWS and bills by ECPU (read/write units) + GB stored. It is advantageous for workloads with unpredictable traffic or frequent short bursts. Downside: the unit price is higher than reserved nodes, so it is a loss for steady traffic.

MemoryDB vs ElastiCache Redis: Cache vs. Primary DB

MemoryDB is a Redis-compatible in-memory DB launched in 2021. It uses the same Redis API as ElastiCache, but the internal architecture differs — every write is synchronously replicated to a multi-AZ transaction log (DynamoDB-based) before the ack is returned. In other words, it provides the same RPO≈0 durability as RDS Multi-AZ at millisecond-level latency.

For a new microservice where "single-entity reads/updates are the main workload + millisecond responses are needed + DB-grade durability," MemoryDB can be more suitable than RDS. The unit price is higher, but it simplifies operating a DB Proxy/replicas/backups. When the keyword combination "microsecond latency + durable" appears, consider MemoryDB.

Summary

ElastiCache looks simple, but decisions pile up across four layers: engine choice (Redis vs Memcached) → topology choice (Cluster Mode On/Off) → caching strategy (Lazy/Write-Through/...) → invalidation mechanism. Each layer's choice affects consistency, performance, and cost differently, so exam scenarios ask "which combination is optimal for the given requirements."

Here are the two decision rules for solving scenarios, restated.

• If you need any one of persistence, Multi-AZ, or complex data structures, choose Redis.
• If there is a risk of stale cache, don't use lazy loading alone. Keep the TTL short and combine it with event-based invalidation or write-through.

In the next post we look at the internal structure of Aurora — a separate engine that keeps only RDS compatibility while completely redesigning the storage and replication layers — and Aurora Serverless v2.