Day 2 - Encryption: KMS and Per-Service Encryption

📌 Key Takeaways

• Encryption at rest and encryption in transit are separate layers. You have to enforce both.
• KMS does not encrypt your data directly. This is envelope encryption: a KMS key wraps a data key, and the data key encrypts the actual data.
• S3 splits into SSE-S3 (applied by default, no key control) / SSE-KMS (key policy, auditing, bucket keys) / DSSE-KMS (dual layer) / SSE-C (customer-provided keys) / client-side (AWS never sees the plaintext).
• The center of KMS access is the key policy. Service roles and cross-account principals must be allowed in the key policy (or a grant).
• Reduce the KMS call cost and throttling of SSE-KMS with an S3 Bucket Key, and block plaintext HTTP with an aws:SecureTransport deny condition.

The Three Kinds of KMS Keys

Data has to be protected both while it is stored and while it moves. At the center of encryption on AWS is KMS (Key Management Service), and data services such as S3, Redshift, and Glue integrate with KMS to use keys.

• CMK (Customer Master Key) in older material is the former term for what is now called a "KMS key." Exam questions may mix both terms.
• The automatic rotation period of AWS managed keys changed from 3 years to 1 year in May 2022. Older material still says 3 years.

How Envelope Encryption Actually Works

Write
 ① The service sends a GenerateDataKey request to KMS
 ② KMS returns the data key in two forms: a plaintext data key + a data key encrypted by the KMS key
 ③ The service encrypts the data with the plaintext data key and discards the plaintext key from memory
 ④ The ciphertext data + the encrypted data key are stored together
 
Read
 ⑤ The service sends the encrypted data key to KMS with a Decrypt request
 ⑥ KMS checks the key policy and returns the plaintext data key
 ⑦ The service decrypts the data with that key

• The KMS key itself never leaves KMS. Only data keys travel in and out.
• Large data is encrypted with the data key on the service side, so there is no latency or cost from sending the whole dataset to KMS.
• The KMS permission the reader needs is kms:Decrypt, and the writer needs kms:GenerateDataKey. "I have S3 permission but can't read an SSE-KMS object" is almost always blocked at step ⑥.

💡 Related theory: KMS does not encrypt the data itself; it manages data keys. That is why rotating a KMS key does not re-encrypt data that is already stored. What changes is the key material used to wrap data keys from then on.

Key Policy: The Center of KMS Access

• A key policy is the resource-based policy of a KMS key. Unlike other resource policies, a key policy does not automatically grant permissions to the account or to principals in the account.
• The default key policy contains a statement that allows kms:* to the account root principal. This statement is what makes it possible to allow key access through IAM policies. Without it, an Allow in an IAM policy has no effect (a Deny still does).
• For cross-account access, the key policy in the key-owning account must allow the other account, and the other account's IAM policy must allow use of that key. Both sides are required.
• A grant is a way to delegate only limited operations to a specific principal. It is mostly created when an AWS service uses a key on a user's behalf.

{
  "Sid": "AllowGlueRoleUseOfKey",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:role/GlueETLRole" },
  "Action": ["kms:Decrypt", "kms:GenerateDataKey"],
  "Resource": "*"
}

• In a key policy, a Resource of * means "this very key the policy is attached to." It does not mean all keys.
• To allow use only when it goes through a specific service, use the kms:ViaService condition. For example, if you allow decryption only through S3, the path where the same role calls the KMS API directly to use the key is closed.

Rotation and Deletion

• Once a key is deleted, data encrypted with that key cannot be recovered. If you are not certain, disable it instead of deleting it. Disabling is reversible.
• A key in its waiting period cannot be used for cryptographic operations. During this period, use CloudTrail and alarms to check who is still using the key.
• Rotation does not reduce the damage from a leaked data key. The reason you need rotation is usually a regulation or contract.

Five Ways to Encrypt S3 at Rest

• Even if you change default encryption to SSE-KMS, existing objects stay as they are. To change existing objects you have to rewrite them by copying (for example with S3 Batch Operations).
• SSE-C is blocked by default on new general purpose buckets starting April 2026. To use it, you have to allow it explicitly in the bucket's encryption settings. Older material describes SSE-C as an option you can use unconditionally.
• AWS services cannot open SSE-C and client-side encrypted objects. Macie reports only the metadata of such objects, and it is also hard to analyze them directly with services like Athena.
• Server-side encryption (SSE) is applied by S3 after it receives the data. At the moment of receipt, the plaintext passes through S3. Only client-side encryption satisfies a requirement that "AWS must never be able to see the plaintext."

{
  "ServerSideEncryptionConfiguration": {
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "aws:kms",
        "KMSMasterKeyID": "arn:aws:kms:ap-northeast-2:111122223333:key/abcd-1234"
      },
      "BucketKeyEnabled": true
    }]
  }
}

How S3 Bucket Key Works and Its Side Effects

• Without a bucket key, S3 requests a data key from KMS for every object. An ETL job that reads millions of objects runs into KMS request charges and request limits.
• With a bucket key enabled, S3 obtains a short-lived bucket-level key from KMS and uses it to generate per-object data keys. Requests to KMS drop sharply, which can cut request costs by up to 99%.
• Control over the KMS key stays the same. That is why, for a requirement of "reduce cost but keep key control," the answer is a bucket key, not a switch to SSE-S3.

Where to Configure Encryption per Service

• Redshift: You can also encrypt an existing unencrypted cluster by changing the setting. Redshift then migrates the data to a new encrypted cluster. RA3 families allow reads and writes while this is in progress, but DC2 allows reads only. The older material's "only possible at creation" and "manage keys with HSM" differ from the current behavior. HSM integration is exclusive to the legacy CloudHSM Classic and is not supported on DC2 and RA3 nodes.
• If you disable a customer managed key for Redshift, the warehouse becomes inaccessible, and if you do not restore the key within 14 days, it is deleted. This is a path where a key operations mistake leads to data loss.
• KMS keys are bound to a Region. If you copy a Redshift snapshot to another Region and want to use your own key in the destination Region, first create a snapshot copy grant in the destination Region.
• Glue security configuration: To encrypt CloudWatch logs with KMS, the key policy must allow the logs service principal, and the job role must have logs:AssociateKmsKey. If it is missing, real-time logging is turned off. A job running inside a VPC fails with a KMS timeout if there is no path to KMS (a KMS VPC endpoint or a NAT).
• Once a security configuration is attached, an SSE-S3 setting passed as a job parameter is ignored. If the two overlap, the security configuration wins.

Encryption in Transit

• AWS API endpoints support TLS. The issue is whether plaintext HTTP is allowed. You enforce this by denying requests where aws:SecureTransport is false in the bucket policy.
• Redshift enforces SSL connections with require_ssl in the parameter group.

{
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": ["arn:aws:s3:::lake-curated", "arn:aws:s3:::lake-curated/*"],
  "Condition": { "Bool": { "aws:SecureTransport": "false" } }
}

💡 Related theory: Encryption at rest and encryption in transit are separate protection layers. Even if storage is encrypted with SSE-KMS, data sent over plaintext HTTP is exposed to man-in-the-middle attacks. In a security review question, if "it's encrypted but it was flagged," look for the missing layer.

Client-Side Encryption

• The application encrypts the data itself before it goes to AWS. You implement it with the AWS Encryption SDK or the S3 encryption client.
• You can wrap the data key with a KMS key (the key is in KMS, encryption and decryption happen on the client), or you can use your own keys.
• AWS receives only ciphertext. It gives the strongest confidentiality, but it makes server-side features (direct analysis by services, sensitive data detection) hard to use and raises operational complexity.

Symptom → Cause → Action

Clue in the Question → Answer Lookup

Tomorrow we cover the problems that encryption and permissions alone can't stop: finding where sensitive data lives (Macie) and reducing exposure (masking and tokenization).

📖 Glossary

• KMS key: The top-level key that stays inside KMS and wraps data keys. The same thing as a CMK in older material.
• Data key: The key that encrypts the actual data. It is stored next to the data in a form encrypted by the KMS key.
• Envelope encryption: A two-tier structure where data is encrypted with a data key and the data key is encrypted with a KMS key.
• Key policy: A resource-based policy attached to a KMS key. It determines who can use and manage the key.
• Grant: A way to delegate only some operations of a key to a specific principal. Mostly created when an AWS service uses the key on your behalf.
• S3 Bucket Key: A bucket-level key that S3 obtains from KMS and holds briefly to generate per-object data keys. It reduces KMS calls.
• Encryption context: Additional information bound in when encrypting. The same value must be present when decrypting, and it can also be used as a policy condition.
• SSE: Server-side encryption. The service encrypts the data after receiving it and before storing it.
• Security configuration: A setting that bundles the encryption methods for the S3 data, logs, and bookmarks a Glue job uses.
• Key rotation: Creating new key material for encryption while leaving the key ID unchanged.