Day 1 - KMS: Handling Encryption Keys Without Touching Them Directly

The first wall a developer learning encryption hits is not the algorithm itself, but the question "where do I put the key?". Encrypting data with AES-256 takes one line of code, but the moment you try to decide where to store that 256-bit key, everything gets complicated. Hardcode it in code and it lives in git forever. Put it in an environment variable and it leaks on process dump. Put it in a file and then who guards that file? The old security industry saying rings true here: "the real problem with encryption is not encryption, but key management." AWS KMS (Key Management Service) solves this "key must never be exported in plaintext" problem with a single design principle.

In the DVA-C02 exam, KMS appears as both a standalone topic and, more importantly, behind the encryption of nearly every storage service — S3, EBS, RDS, DynamoDB, Secrets Manager. Once you understand KMS's operation model — especially Envelope Encryption and the three-tier permission structure of key policies — half of the exam's security section solves itself. This article looks at why KMS was designed to keep keys inside, where the 4KB limit comes from, and how envelope encryption actually works to bypass that limit.

The Problem KMS Set Out to Solve: Keys Must Never Leave the Boundary

The core design decision of KMS is surprisingly simple. The key material of a CMK (Customer Master Key) can never be extracted as plaintext via any API. Call kms:Encrypt and KMS encrypts internally, returning only the ciphertext. Call kms:Decrypt and KMS decrypts internally, returning only the plaintext. The key itself exists only within the HSM (Hardware Security Module) boundary and never crosses it. This is the fundamental difference between KMS and traditional PKI tools where you create a key and download it as a file.

💡 Related theory: This design implements the cryptographic principle of key isolation at cloud scale. FIPS 140-2 defines security levels 1-4 for cryptographic modules, and "keys do not cross the module boundary in plaintext" is a core requirement for Level 2 and above. KMS uses HSM validated to FIPS 140-2 Level 3 (on multi-tenant HSM, in some regions as of 2023). Making keys non-extractable is not sacrificing convenience but a direct consequence of the threat model: "what cannot be extracted cannot be exfiltrated."

🔍 Going deeper: Intentionally, there is no GetKeyMaterial function in the KMS API. The only direction to retrieve keys is the opposite one — the Imported Key Material feature only lets you put your key into KMS. Even then, you must encrypt the key with KMS's public key (obtained from GetParametersForImport) before sending it, and once it's in, you cannot get it back out. In BYOK (Bring Your Own Key) scenarios where AWS cannot see the key, this one-directional property is preserved. Users keep the original in an external HSM and can delete the KMS copy at any time — this control is important for compliance customers.

This "keys never escape" constraint becomes KMS's biggest limitation. Since every encryption and decryption must go through the KMS API call, directly encrypting large data via KMS means ① sending data over the network to KMS ② receiving it back. That's why KMS limits the maximum plaintext size that can be directly encrypted to 4KB.

⚠️ Trap: "4KB" appears in almost every exam question. The maximum plaintext you can directly encrypt with kms:Encrypt is 4,096 bytes. Missing this leads to the trap of choosing direct kms:Encrypt for "how to encrypt a 10MB file with KMS." The answer is always envelope encryption. The 4KB limit reflects the design intent that KMS is "a tool to protect small secrets (data keys, passwords, tokens)" not "a tool to directly encrypt large data."

Envelope Encryption: A Two-Step Trick of Encrypting Keys with Keys

The elegant way to bypass the 4KB limit is to not ask KMS to encrypt the data directly. Instead, create one disposable Data Encryption Key (DEK), then encrypt large data locally with that key. KMS's master key (CMK) serves only one purpose: protecting (encrypting) that data key. Keys wrap around keys — hence the term "envelope" encryption.

import boto3, os
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
 
kms = boto3.client('kms')
 
# 1) Request one data key from KMS — receive both plaintext DEK and encrypted DEK simultaneously
resp = kms.generate_data_key(
    KeyId='alias/myapp-key',
    KeySpec='AES_256'
)
plaintext_dek  = resp['Plaintext']        # 32 bytes plaintext key (used for encryption)
encrypted_dek  = resp['CiphertextBlob']   # Key encrypted with CMK (stored with data)
 
# 2) Encrypt large data locally with the plaintext DEK (no KMS call)
aesgcm = AESGCM(plaintext_dek)
nonce  = os.urandom(12)
ciphertext = aesgcm.encrypt(nonce, b'...10MB of data...', None)
 
# 3) Immediately discard plaintext DEK from memory — this is the key point
del plaintext_dek
 
# 4) Storage: [encrypted data + nonce + encrypted DEK]
#    The encrypted DEK can only be decrypted by going through KMS Decrypt

Decryption works exactly the reverse. Take the stored encrypted DEK, decrypt it with kms:Decrypt to get plaintext DEK briefly, decrypt the data locally with it, and discard the plaintext DEK again.

💡 Related theory: This pattern did not originate with KMS but comes from the hybrid encryption lineage that PGP/GPG has used since 1991. PGP rapidly encrypts email with a symmetric key (session key) and encrypts only that session key with the recipient's RSA public key. Symmetric encryption is fast but key exchange is hard; asymmetric is easy to exchange but slow — hybrid combines them: "large data symmetric, small keys asymmetric (or protected master key)." Envelope encryption applies this idea to cloud key management, and S3 SSE-KMS, EBS volume encryption, and RDS encryption all use this internally.

🔍 Going deeper: The key insight is that GenerateDataKey returns both plaintext DEK and encrypted DEK in a single call. If they were separate, you'd need an API to fetch the plaintext DEK, which violates KMS's "never export keys plaintext" rule. AWS resolved this contradiction by deciding "data keys are not CMK" — the DEK is a one-time dependent key generated on the spot by the CMK. Being brief-lived and disposable, the plaintext DEK can briefly escape without breaking CMK isolation. The plaintext DEK must never touch disk because if it does, a plaintext key sits next to encrypted data, rendering encryption meaningless. That's why GenerateDataKeyWithoutPlaintext exists as a variant — receive only the encrypted DEK to store ahead, then plaintext-ify only at decryption time.

📚 Case study: S3 SSE-KMS workloads handling millions of objects hit KMS API limits if calling GenerateDataKey per object. AWS's S3 Bucket Key (introduced 2020) solved this by adding one more layer to envelope encryption — creating a bucket-level key cached briefly to reduce per-object DEK generation KMS calls by 99%. Stacking the "keys wrap keys" idea one layer deeper, it dramatically lowered KMS costs in bulk-object environments.

Key Types: Who Owns and Controls the Key

KMS keys split into three types based on who creates and controls the policy. This distinction is a frequent exam trap.

⚠️ Trap: AWS Managed Key auto-rotation period is 365 days (1 year). Pre-May 2022 materials say "3 years (1095 days)"; AWS changed policy and now it's 1 year. Old questions may linger in test banks, so pick 1 year when "AWS Managed Key rotation period" appears. CMK auto-rotation also defaults to 1 year, and since 2024, custom periods (90–2560 days) are configurable.

🔍 Going deeper: Key rotation does not "create a new key" but rather adds new key material (backing key). Rotating a CMK keeps the ARN, Key ID, and alias — only the backing key changes. New encryptions use the new backing key; already-encrypted data auto-decrypts with its original backing key because KMS records which backing key was used in the ciphertext header. Post-rotation, old data does not need re-encryption, and application code seeing only the ARN changes nothing. This answers the common misunderstanding "won't old data become unreadable after rotation?"

Key Policy + IAM + Grant: KMS's Three-Tier Permission Model

The decisive point where KMS differs from other AWS services is that key policy is the final authority on permissions. Most services control access via IAM policy alone, but KMS adds a key policy on the key itself that decides "does this key delegate to IAM permissions at all?"

{
  "Sid": "Enable IAM User Permissions",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:root" },
  "Action": "kms:*",
  "Resource": "*"
}

If this statement is absent from the key policy, even if IAM policy allows kms:Decrypt, it has no effect. Allowing the root principal means "I grant this account's IAM policy decision-making authority"; without it, only principals explicitly named in the key policy can use the key.

⚠️ Trap: "I gave kms:* in IAM policy but access is denied?" is a frequent exam scenario. Answer: the key policy lacks the IAM delegation statement. The default key policy includes this statement automatically, but writing a custom policy and omitting it results in "key lockout" — even the creator cannot use the key. So the answer is always "both key policy AND IAM policy required," never "IAM policy alone is enough."

The third axis, Grant, is a code-based way to grant temporary, fine-grained permissions. Without touching Key Policy or IAM, issue one-off permissions like "this principal can only Decrypt with this key, under this condition."

🔍 Going deeper: Grant is often used internally when AWS services must use a key on the user's behalf. For example, attaching an encrypted EBS volume to EC2 causes the EC2 service to obtain a temporary Grant for that volume's CMK to decrypt at boot. Users do not need to modify key policies directly; the service creates minimal-permission Grants as needed, then deletes them when done. Grant is an excellent pattern for "least privilege, short-lived" enforcement when Lambda must temporarily use a key on behalf of another user.

KMS API Limits and Asymmetric Keys

A single CMK's encryption API throughput is limited per region and key type (symmetric keys typically support thousands per second; large regions like us-east-1 go higher). Exceeding the limit raises ThrottlingException. This is the second reason envelope encryption reduces KMS calls — reusing and caching DEK avoids API explosion.

KMS supports both symmetric and asymmetric keys.

⚠️ Trap: ECC (elliptic curve) keys can only sign and verify, not encrypt. ECDSA is a signing algorithm, not an encryption algorithm. If you need asymmetric encryption, pick RSA. Also, asymmetric keys can export the public key via GetPublicKey, enabling external systems to encrypt with the public key and KMS to decrypt with the private key — unlike symmetric keys, the public key is not isolation-protected.

Multi-Region Key and Key Deletion Safeguards

By default, a CMK is region-bound. Data encrypted in ap-northeast-2 cannot be decrypted by KMS in another region. For multi-region workloads (S3 Cross-Region Replication, DynamoDB Global Tables), Multi-Region Key maintains the same key ID across replicas in multiple regions, allowing decryption in a different region of data encrypted in one.

Key deletion has strong safeguards. Calling ScheduleKeyDeletion does not delete immediately; instead, a 7–30 day waiting period begins. During this time, the key is PendingDeletion (unusable) but can be cancelled via CancelKeyDeletion.

💡 Related theory: Making keys non-immediately-deletable exemplifies a safety-design pattern: "add time delay to irreversible destructive operations." Deleting a CMK permanently makes all data encrypted with it irrecoverable — backups are useless. A 7–30 day grace window provides a window to undo accidents or malice. The same philosophy underlies S3 object version MFA Delete and RDS final snapshot requirements. Note: there is no immediate-deletion option at all.

CloudHSM and KMS's Boundary: When KMS Isn't Enough

KMS runs on multi-tenant HSM — multiple customers' keys are logically isolated but share physical hardware. Heavily regulated industries (certain financial, government) demand "hardware dedicated to my keys only." Enter CloudHSM.

🔍 Going deeper: KMS can connect CloudHSM as a custom key store. This means key material lives in the user's dedicated CloudHSM cluster while remaining accessible via the familiar KMS API — the actual crypto operations happen on dedicated hardware. Combining "KMS convenience + CloudHSM dedicated isolation." If "keys must be on AWS-invisible dedicated HW" in a regulatory context appears on the exam, CloudHSM (or KMS custom key store) is the answer; general "managed key encryption" is KMS.

Hands-On with CLI

# 1) Create CMK + assign alias
aws kms create-key --description "myapp prod key"
aws kms create-alias --alias-name alias/myapp-key --target-key-id <key-id>
 
# 2) Enable auto rotation (1 year)
aws kms enable-key-rotation --key-id alias/myapp-key
aws kms get-key-rotation-status --key-id alias/myapp-key
 
# 3) Issue data key for envelope encryption
aws kms generate-data-key --key-id alias/myapp-key --key-spec AES_256
 
# 4) Direct encryption for ≤4KB (e.g., config token)
aws kms encrypt --key-id alias/myapp-key --plaintext fileb://token.bin --output text --query CiphertextBlob
 
# 5) Schedule key deletion (minimum 7 days' wait)
aws kms schedule-key-deletion --key-id <key-id> --pending-window-in-days 7

Wrapping Up

KMS's problem was "keep keys in plaintext out while making encryption convenient." That constraint created the 4KB limit, and bypassing it with envelope encryption became the standard for all AWS storage encryption. Key policy is the final permission authority; missing the IAM delegation statement locks the key. Key deletion has 7–30 days' grace. ECC handles signing only; RSA handles encryption. These subtle differences are exam trap cores.

Next we look at Secrets Manager and Parameter Store, layering on top of KMS to handle secret storage and even automatic rotation.