Day 3 - Nested Stack, Cross-Stack Reference, StackSets: Large-Scale IaC Operations

Within Amazon, the "Two-Pizza Team" principle emerged in the mid-2000s: team size should be limited to what can be fed by two pizzas (5-8 people), and each team independently owns and deploys its service. This principle became the seed of microservices architecture.

CloudFormation faces the same challenge. A single template with thousands of lines exceeds the scope of "one team understands and owns everything." We need a way to separate IaC by team boundaries, safely share information between teams, and deploy standards across the entire organization. Nested Stack, Cross-Stack Reference, and StackSets solve this problem. These three tools address problems at different scales: "modularization within one application," "resource sharing across teams," and "standard deployment across the organization."

Nested Stack: "Building Infrastructure Like LEGO Blocks"

Nested Stack is a pattern where one Stack contains another Stack as a resource (AWS::CloudFormation::Stack). It separates large templates into small components to increase reusability and limit the scope of impact when components change.

Technical Operation of Nested Stack:

When a parent Stack creates a resource of type AWS::CloudFormation::Stack, CloudFormation fetches the Template from the S3 URL and creates a separate Stack. Parent and child Stacks have separate Stack IDs but are bound to the parent's lifecycle. When the parent Stack is deleted, child Stacks are deleted in reverse order (opposite of dependency order).

Hierarchical Design Example:

Root Stack (parent.yaml)
├── NetworkStack (network.yaml) - VPC, Subnet, IGW, NAT, RouteTables
│   └── Outputs: VpcId, PublicSubnetIds, PrivateSubnetIds, NatGwEips
│
├── SecurityStack (security.yaml) - Standard Security Groups set
│   └── Outputs: WebSgId, AppSgId, DbSgId
│
├── DatabaseStack (database.yaml) - RDS, ElastiCache
│   └── Inputs: VpcId, PrivateSubnetIds, DbSgId
│   └── Outputs: DbEndpoint, CacheEndpoint
│
└── AppStack (app.yaml) - ALB, ECS, Auto Scaling
    └── Inputs: VpcId, PublicSubnetIds, WebSgId, AppSgId, DbEndpoint

Real-World Parent Template:

AWSTemplateFormatVersion: '2010-09-09'
Description: Root Stack - Production Web Application
 
Parameters:
  TemplateBucket:
    Type: String
    Default: my-cfn-templates-123456789012
 
Resources:
  NetworkStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub 'https://s3.${AWS::URLSuffix}/${TemplateBucket}/network.yaml'
      Parameters:
        VpcCidr: 10.0.0.0/16
        Environment: prod
      TimeoutInMinutes: 20
      Tags:
        - Key: Component
          Value: Network
 
  SecurityStack:
    Type: AWS::CloudFormation::Stack
    DependsOn: NetworkStack      # Technically unnecessary since dependency is expressed via !GetAtt, but explicit for clarity
    Properties:
      TemplateURL: !Sub 'https://s3.${AWS::URLSuffix}/${TemplateBucket}/security.yaml'
      Parameters:
        VpcId: !GetAtt NetworkStack.Outputs.VpcId
 
  DatabaseStack:
    Type: AWS::CloudFormation::Stack
    DeletionPolicy: Retain       # Prevent accidental deletion of RDS in child Stack
    Properties:
      TemplateURL: !Sub 'https://s3.${AWS::URLSuffix}/${TemplateBucket}/database.yaml'
      Parameters:
        VpcId: !GetAtt NetworkStack.Outputs.VpcId
        SubnetIds: !GetAtt NetworkStack.Outputs.PrivateSubnetIds
        DbSgId: !GetAtt SecurityStack.Outputs.DbSgId
 
  AppStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: !Sub 'https://s3.${AWS::URLSuffix}/${TemplateBucket}/app.yaml'
      Parameters:
        VpcId: !GetAtt NetworkStack.Outputs.VpcId
        PublicSubnetIds: !GetAtt NetworkStack.Outputs.PublicSubnetIds
        WebSgId: !GetAtt SecurityStack.Outputs.WebSgId
        DbEndpoint: !GetAtt DatabaseStack.Outputs.DbEndpoint
 
# Parent exposes child Outputs to the outside
Outputs:
  AppUrl:
    Value: !GetAtt AppStack.Outputs.AlbDnsName

Template Size Limits:

If Template exceeds 1 MB, either split into Nested Stacks or use CDK.

# Upload template to S3 and use URL
aws s3 sync ./templates/ s3://my-cfn-templates-123456789012/
 
aws cloudformation create-stack \
  --stack-name webapp-root \
  --template-url "https://s3.ap-northeast-2.amazonaws.com/my-cfn-templates-123456789012/parent.yaml" \
  --parameters ParameterKey=TemplateBucket,ParameterValue=my-cfn-templates-123456789012 \
  --capabilities CAPABILITY_NAMED_IAM
 
# Update child Stack directly (without parent)
aws cloudformation update-stack \
  --stack-name webapp-root-NetworkStack-XXXX \
  --template-url "https://s3.ap-northeast-2.amazonaws.com/my-cfn-templates-123456789012/network.yaml" \
  --parameters ParameterKey=VpcCidr,ParameterValue=10.0.0.0/16

💡 Related Theory: Nested Stack applies the software engineering principle of "Modularization" to infrastructure. David Parnas's 1972 paper "On the Criteria To Be Used in Decomposing Systems into Modules" defines the core principle: "Hide parts likely to change in one module." Even if the network team changes VPC CIDR, the application team's template remains unaffected because the change scope is encapsulated within NetworkStack. This is the same concept as information hiding in object-oriented programming. Nested Stack's Outputs are public interfaces, while internal implementation (subnet division, NAT GW count) remains hidden.

🔍 Deeper Dive: The parent-child relationship in Nested Stack is a direct hierarchy without dependency inversion. Since the parent accesses child Outputs, changing child Output names requires updating the parent template. To prevent this, define child Stack Output names as "stable interfaces" and avoid changing them. In production deployment, patterns use versioned S3 directories: s3://my-bucket/templates/v1.2.0/network.yaml. This enables Blue-Green style deployment: upload new template version to S3, test, then update parent Stack to switch.

⚠️ Pitfall: DeletionPolicy on Nested Stack applies to Stack resources, not the Stack itself. Setting DeletionPolicy: Retain on DatabaseStack resource preserves the Stack itself, not the resources inside (RDS, ElastiCache). To protect RDS inside child Stack, also set DeletionPolicy: Snapshot on the RDS resource in the child template.

Cross-Stack Reference: "Independent Yet Connected"

Cross-Stack Reference shares values between completely independent Stacks. Unlike Nested Stack, there's no parent-child relationship, and each Stack has independent create/update/delete lifecycle. It's well-suited for resource sharing across team boundaries.

Export/Import Mechanism:

# Stack A (network-stack): Create VPC then Export
Outputs:
  VpcId:
    Value: !Ref MyVpc
    Description: Shared VPC ID for all application stacks
    Export:
      Name: !Sub '${AWS::StackName}-VpcId'  # → network-stack-VpcId
 
  PrivateSubnetIds:
    Value: !Join [',', [!Ref PrivateSubnetA, !Ref PrivateSubnetB, !Ref PrivateSubnetC]]
    Export:
      Name: !Sub '${AWS::StackName}-PrivateSubnetIds'
    # → network-stack-PrivateSubnetIds = "subnet-aaa,subnet-bbb,subnet-ccc"
 
  DbSubnetGroupName:
    Value: !Ref DbSubnetGroup
    Export:
      Name: !Sub '${AWS::StackName}-DbSubnetGroupName'

# Stack B (web-app-stack): Import from network-stack
Resources:
  WebServer:
    Type: AWS::EC2::Instance
    Properties:
      # Select first subnet from comma-separated list
      SubnetId: !Select [0, !Split [',', !ImportValue 'network-stack-PrivateSubnetIds']]
 
  AppDatabase:
    Type: AWS::RDS::DBInstance
    Properties:
      DBSubnetGroupName: !ImportValue 'network-stack-DbSubnetGroupName'
      VpcSecurityGroupIds:
        - !ImportValue 'security-stack-DbSgId'

Key Constraints and Operational Impact of Cross-Stack:

# Find which Stacks use a specific Export
aws cloudformation list-imports \
  --export-name "network-stack-VpcId"
 
# List all Exports
aws cloudformation list-exports \
  --query 'Exports[*].[Name,ExportingStackId,Value]' \
  --output table
 
# If Export change needed: first identify importing Stacks
IMPORTERS=$(aws cloudformation list-imports \
  --export-name "network-stack-VpcId" \
  --query 'Imports[]' --output text)
echo "Stacks using this Export: $IMPORTERS"
# Must modify these Stacks first before changing network-stack VpcId Export

🔍 Deeper Dive: Cross-Stack's "cannot change Export while imported" constraint is identical to the "breaking change" problem in distributed systems. Just as an API server cannot change endpoints while clients use them, the same principle applies here. The workaround is the "versioned Export name" pattern: network-stack-VpcId-v1, network-stack-VpcId-v2. During migration, export both versions simultaneously. Consumer Stacks migrate to new version, then delete old. This mirrors API versioning with "Sunset Date" strategy: publish new version, give users migration window, then delete old.

Nested Stack vs Cross-Stack Selection Criteria:

StackSets: "Deploy Organization-Wide in One Go"

StackSets deploy a single template to multiple AWS accounts in multiple regions simultaneously. It's a critical tool for implementing organization-wide standards like Landing Zones, security baselines, and monitoring standardization.

StackSets Architecture:

Management Account
    │
    └── StackSet "OrgSecurityBaseline"
            │ (one template, many targets)
            │
    ┌───────┼───────┬──────────┐
    ▼       ▼       ▼          ▼
 Account A Account B Account C Account D
 (Seoul,   (Seoul,  (Seoul    (Seoul,
  Virginia) Virginia) only)    Virginia,
                                Tokyo)
    │
    └── Stack Instance (one per account+region combination)
        Account A/Seoul: Stack Instance 1
        Account A/Virginia: Stack Instance 2
        ...

Permission Model: Self-Managed vs Service-Managed

Self-Managed Permissions:

# 1. Create AdministrationRole in management account
aws iam create-role \
  --role-name AWSCloudFormationStackSetAdministrationRole \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"Service": "cloudformation.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }]
  }'
 
# 2. Grant permissions to AdministrationRole (Assume ExecutionRole)
aws iam attach-role-policy \
  --role-name AWSCloudFormationStackSetAdministrationRole \
  --policy-arn arn:aws:iam::aws:policy/AdministratorAccess
 
# 3. Create ExecutionRole in each Target account
# (AdministrationRole in management account assumes this Role)
aws iam create-role \
  --role-name AWSCloudFormationStackSetExecutionRole \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::MANAGEMENT_ACCOUNT_ID:root"},
      "Action": "sts:AssumeRole"
    }]
  }'

Service-Managed Permissions (Organizations integration, recommended):

# Enable StackSets Trusted Access in Organizations (run once in management account)
aws organizations enable-aws-service-access \
  --service-principal stacksets.cloudformation.amazonaws.com
 
# Create StackSet (SERVICE_MANAGED)
aws cloudformation create-stack-set \
  --stack-set-name "OrgSecurityBaseline" \
  --template-body file://security-baseline.yaml \
  --permission-model SERVICE_MANAGED \
  --auto-deployment 'Enabled=true,RetainStacksOnAccountRemoval=false' \
  --capabilities CAPABILITY_NAMED_IAM \
  --description "Org-wide: CloudTrail, GuardDuty, Config, SecurityHub"

AutoDeployment=true is key. When a new AWS account is added to an OU, an EventBridge event fires and CloudFormation automatically creates Stack Instance. With RetainStacksOnAccountRemoval=false, Stack Instance is deleted when account is removed from OU.

Stack Instance Deployment:

# Deploy to entire OU (multi-region, parallel)
aws cloudformation create-stack-instances \
  --stack-set-name "OrgSecurityBaseline" \
  --deployment-targets '{
    "OrganizationalUnitIds": ["ou-root-abc123", "ou-workloads-xyz456"]
  }' \
  --regions ap-northeast-2 us-east-1 eu-west-1 \
  --operation-preferences '{
    "RegionConcurrencyType": "PARALLEL",
    "MaxConcurrentPercentage": 25,
    "FailureTolerancePercentage": 10
  }'
 
# Deploy to all OU except specific account
aws cloudformation create-stack-instances \
  --stack-set-name "OrgSecurityBaseline" \
  --deployment-targets '{
    "OrganizationalUnitIds": ["ou-workloads-xyz"],
    "AccountFilterType": "DIFFERENCE",
    "Accounts": ["111122223333"]
  }' \
  --regions ap-northeast-2
 
# Update StackSet (apply new template to all Stack Instances)
aws cloudformation update-stack-set \
  --stack-set-name "OrgSecurityBaseline" \
  --template-body file://security-baseline-v2.yaml \
  --operation-preferences '{
    "MaxConcurrentPercentage": 10,
    "FailureTolerancePercentage": 5,
    "RegionOrder": ["ap-northeast-2", "us-east-1", "eu-west-1"]
  }'

Operation Preferences Options:

📚 Case Study: In 2022, fintech company H restructured its account hierarchy with AWS Organizations and built an "org-wide security baseline" using StackSets. It included CloudTrail (all API logging), Config (baseline rules), GuardDuty (threat detection), and SecurityHub (unified security dashboard) deployed to all accounts. With Organizations + Service-Managed StackSets + AutoDeployment=true, security baseline automatically applied to new accounts instantly. Previous 1-2 days of manual setup became zero minutes. SOC2 Type II audit confirmed "identical security controls applied to all accounts" via StackSets operational records.

StackSet Operations and Management:

# Query Stack Instance status
aws cloudformation list-stack-instances \
  --stack-set-name "OrgSecurityBaseline" \
  --query 'Summaries[*].[Account,Region,Status,StatusReason]' \
  --output table
 
# Stack Instance status meanings:
# CURRENT: In sync with latest StackSet
# OUTDATED: StackSet updated but this Instance not yet applied
# INOPERABLE: Unrecoverable state (manual cleanup needed)
 
# StackSet Drift Detection (all Stack Instances simultaneously)
aws cloudformation detect-stack-set-drift \
  --stack-set-name "OrgSecurityBaseline" \
  --operation-preferences 'MaxConcurrentPercentage=10'
 
# Retry only failed Stack Instances
aws cloudformation update-stack-instances \
  --stack-set-name "OrgSecurityBaseline" \
  --deployment-targets 'Accounts=["111122223333"]' \
  --regions ap-northeast-2 \
  --operation-preferences 'MaxConcurrentCount=1,FailureToleranceCount=0'
 
# StackSet deletion order: Delete Stack Instances first, then StackSet
aws cloudformation delete-stack-instances \
  --stack-set-name "OrgSecurityBaseline" \
  --deployment-targets 'OrganizationalUnitIds=["ou-workloads-xyz"]' \
  --regions ap-northeast-2 us-east-1 \
  --no-retain-stacks  # Also delete actual Stacks in Stack Instances
 
aws cloudformation delete-stack-set \
  --stack-set-name "OrgSecurityBaseline"
# Error: Cannot delete if Stack Instances remain

DependsOn: Explicit Dependency Relationships

CloudFormation automatically detects dependencies via !Ref or !GetAtt. However, when A must logically follow B but there's no code reference, use DependsOn.

# Create EC2 after VPC Endpoints
# (EC2 needs VPC Endpoints to communicate with SSM)
WebServer:
  Type: AWS::EC2::Instance
  DependsOn:
    - S3GatewayEndpoint
    - SSMInterfaceEndpoint
  Properties:
    ImageId: !Ref AmiId
    SubnetId: !Ref PrivateSubnet
 
S3GatewayEndpoint:
  Type: AWS::EC2::VPCEndpoint
  Properties:
    ServiceName: !Sub 'com.amazonaws.${AWS::Region}.s3'
    VpcId: !Ref MyVpc
    RouteTableIds: [!Ref PrivateRouteTable]
 
SSMInterfaceEndpoint:
  Type: AWS::EC2::VPCEndpoint
  Properties:
    ServiceName: !Sub 'com.amazonaws.${AWS::Region}.ssm'
    VpcId: !Ref MyVpc
    VpcEndpointType: Interface
    SubnetIds: [!Ref PrivateSubnet]
    PrivateDnsEnabled: true

⚠️ Pitfall: Overusing DependsOn serializes creation and increases Stack creation time. CloudFormation naturally creates resources with no dependencies in parallel. Unnecessary DependsOn breaks this parallelism. Dependencies expressible via !Ref or !GetAtt don't need DependsOn. For example, if you reference SG ID via !GetAtt SG.GroupId, CloudFormation automatically creates SG before EC2. Adding DependsOn: SG on top is redundant.

StackSets and SCP Interaction: Important Operational Pitfall

Important caution when SCP (Service Control Policy) and StackSets are used together.

Scenario: SCP forbids resource creation in a specific region (aws:RequestedRegion condition), but StackSets tries to deploy to that region?

Answer: Even if StackSets assumes ExecutionRole, SCP restrictions apply. SCP applies to all IAM entities in the account root, so CloudFormation operations via ExecutionRole are also subject to SCP. Stack Instance creation fails and may halt entire deployment depending on FailureTolerancePercentage.

SCP + StackSets Safety Checklist:

# Before deployment: Verify target region is allowed by SCP
# Simulate whether ap-northeast-2 is allowed in SCP
aws iam simulate-principal-policy \
  --policy-source-arn arn:aws:iam::TARGET_ACCOUNT_ID:role/AWSCloudFormationStackSetExecutionRole \
  --action-names cloudformation:CreateStack \
  --resource-arns "*" \
  --context-entries 'ContextKeyName=aws:RequestedRegion,ContextKeyValues=["ap-northeast-2"],ContextKeyType=string'

💡 Related Theory: The combination of SCP and IAM policies follows AWS permission evaluation's "minimum intersection" principle. Effective permission = (SCP allows) ∩ (Permission Boundary allows) ∩ (Identity Policy allows). This follows set theory intersection where if any denies, result is deny. When designing StackSets, pre-simulate with IAM Policy Simulator to ensure deployment regions and operations fall within SCP allowance. This is mandatory.

📚 Case Study: In 2023, financial institution I restricted all resource creation outside ap-northeast-2 (Seoul) and us-east-1 (Virginia) via SCP. Later, StackSets attempted eu-west-1 (Ireland) deployment to meet European regulations. SCP constraints caused all Stack Instance creations to fail across hundreds of accounts. Repeated deploy→fail cycles caused CloudTrail event explosion. Solution: Isolated accounts needing eu-west-1 into separate OU and added eu-west-1 allowance to that OU's SCP only.

CloudFormation Modules and Registry: Organizational Standard Types

AWS CloudFormation Modules package multiple resources into a reusable unit. Organizations can create modules implementing security standards so all teams use Type: MyOrg::Security::HardenedEC2 the same way.

# Module usage example (custom type)
Resources:
  WebServer:
    Type: MyOrg::Security::HardenedEC2Instance  # Custom module
    Properties:
      InstanceType: t3.medium
      SubnetId: !Ref PrivateSubnet
      # Internally includes IMDSv2 enforcement, CW Agent install, standard SG, encrypted EBS, etc.

Modules register in CloudFormation Registry, then organizations deploy modules to all accounts via StackSets, and each team uses them. This realizes "Governance as Code."

Big Picture: When to Use What?

CloudFormation Pattern Selection Guide
============================================================

Q1: Want to modularize one application infrastructure within team
    └── Nested Stack (store component templates in S3)
        ★ Parent controls full lifecycle

Q2: Want to share resources (VPC, IAM) across multiple teams
    └── Cross-Stack Reference (Outputs.Export / !ImportValue)
        ★ Each Stack has independent lifecycle, loose coupling

Q3: Want to deploy security/monitoring standard to all org accounts
    └── StackSets (Service-Managed + Auto-deployment)
        ★ Organizations integration, new accounts auto-included

Q4: Want multiple teams to easily reuse pattern via standard type
    └── CFn Modules + Registry (type registry)
        ★ Governance as Code

Q5: Want to provide standard infrastructure to developers via self-service
    └── Service Catalog (productize CFn templates)
        ★ Catalog-based self-service (covered in Week 6 Day 4)