Day 4 - IaC & Workflow Orchestration: CloudFormation/CDK, Step Functions, EventBridge, Airflow (MWAA)

📌 Key Takeaways

• IaC (CloudFormation/CDK) defines infrastructure as declarative code, guaranteeing idempotency, reproducibility, and environment consistency.
• CloudFormation uses YAML/JSON templates, while CDK uses Python or TypeScript code → synthesizing (synth) it produces CloudFormation.
• Step Functions is state machine orchestration spanning multiple AWS services such as Lambda, Glue, SageMaker, and SNS.
• EventBridge is not a workflow but a trigger — the glue that starts a workflow on a schedule (cron) or an event.
• MWAA (managed Airflow) is the option when you have existing Airflow assets, a hybrid setup, or complex DAGs.

IaC: CloudFormation and CDK

The pipelines, Registry, and CI/CD so far all rested on the premise that "infrastructure is laid down somewhere." If that infrastructure is built by clicking by hand in the console, it cannot be reproduced and drift appears between environments.

• IaC defines infrastructure as declarative code. The same code stamps out dev/staging/prod identically.
• Change history is tracked with Git, and when a problem occurs you roll back per stack.

With CloudFormation, you write the "desired state" in a template and it builds it as a stack. An example of defining a SageMaker endpoint as IaC:

# CloudFormation: declare a SageMaker endpoint as code
Resources:
  MyModel:
    Type: AWS::SageMaker::Model
    Properties:
      ExecutionRoleArn: !Ref SageMakerRoleArn
      PrimaryContainer:
        Image: !Ref InferenceImageUri
        ModelDataUrl: !Ref ModelArtifactS3Uri
  MyEndpointConfig:
    Type: AWS::SageMaker::EndpointConfig
    Properties:
      ProductionVariants:
        - ModelName: !GetAtt MyModel.ModelName
          VariantName: AllTraffic
          InstanceType: ml.m5.large
          InitialInstanceCount: 1
  MyEndpoint:
    Type: AWS::SageMaker::Endpoint
    Properties:
      EndpointConfigName: !GetAtt MyEndpointConfig.EndpointConfigName

CDK writes the same thing as code (an advantage for loops, conditions, and abstraction).

# AWS CDK (Python): synthesizing generates CloudFormation equivalent to the above
from aws_cdk import aws_sagemaker as sagemaker
 
model = sagemaker.CfnModel(self, "MyModel",
    execution_role_arn=role_arn,
    primary_container={"image": image_uri, "modelDataUrl": model_s3_uri},
)

• The three resources are tied together by references in the order Model → EndpointConfig → Endpoint. The reference is the creation order.
• Deployment is atomic per stack. If it fails midway, CloudFormation rolls back to the previous state.
• If you parameterize the model artifact path and image URI with !Ref, you can redeploy by swapping only the model version in the same template. This is exactly what the Deploy pipeline does.
• Once you start managing infrastructure with IaC, the rule is do not touch it in the console. A change made there is silently overwritten at the next stack deployment.

💡 Related theory: The core value of IaC is idempotency and reproducibility. Applying the same template any number of times ends in the same resulting state, and drift between environments disappears. This is also why SageMaker Projects uses CloudFormation internally. In the exam, "manage infrastructure as reproducible code" means CloudFormation/CDK.

Step Functions — Orchestration Across AWS Services

SageMaker Pipelines centers on SageMaker jobs. AWS Step Functions, on the other hand, orchestrates workflows that span multiple AWS services — Lambda, Glue, SageMaker, ECS, SNS, and others — as a state machine. If an ML workflow contains many steps outside SageMaker (for example, notification via Lambda, ETL via Glue, updating DynamoDB), Step Functions is a good fit.

{
  "Comment": "State machine linking ETL → training → notification",
  "StartAt": "GlueETL",
  "States": {
    "GlueETL": { "Type": "Task", "Resource": "arn:aws:states:::glue:startJobRun.sync", "Next": "TrainModel" },
    "TrainModel": { "Type": "Task", "Resource": "arn:aws:states:::sagemaker:createTrainingJob.sync", "Next": "Notify" },
    "Notify": { "Type": "Task", "Resource": "arn:aws:states:::sns:publish", "End": true }
  }
}

• For SageMaker, the .sync integration lets you wait until the job completes. Without it, the training job is only started and execution moves on to the next state immediately.
• There are two workflow types. Standard suits long-running workflows and keeps an execution history, making it strong for auditing and debugging. Express suits very short, high-volume event processing. ML training orchestration takes a long time, so it is usually Standard.
• Telling Pipelines and Step Functions apart: if you only bundle SageMaker jobs and need lineage and Registry integration, use Pipelines; if you link many services broadly and need general workflow control, use Step Functions.

EventBridge — Event-Based Triggers

Amazon EventBridge is an event bus that receives events and routes them to targets according to rules. Its uses in ML:

• Schedule triggers: Periodic retraining with a cron/rate expression (for example, running the pipeline every day before dawn).
• Event triggers: Starting a workflow on events such as a model approval (what you learned yesterday), new data arriving in S3, or a training job completing.

import boto3
events = boto3.client("events")
 
# A schedule rule that triggers the retraining pipeline every day at 02:00 (UTC)
events.put_rule(
    Name="nightly-retrain",
    ScheduleExpression="cron(0 2 * * ? *)",
)
# (Connect a Lambda that runs the SageMaker Pipeline, or Step Functions, as the target)

EventBridge is the glue that "starts a workflow when something happens (or at a set time)." It does not define the workflow itself; it sends the start signal.

   [New data in S3]  [Model Approved]   [Every day 02:00]
        │                │                │
        └────────────────┼────────────────┘
                         ▼
                  EventBridge rule
              (pattern matching / schedule)
                         │  target specified
        ┌────────────────┼────────────────┐
        ▼                ▼                ▼
 SageMaker Pipeline  Step Functions   CodePipeline
   (start run)        (state machine)  (start deploy)

Amazon MWAA — Managed Apache Airflow

Amazon MWAA (Managed Workflows for Apache Airflow) is managed Airflow. It defines workflows as DAGs written in Python and uses Airflow's rich operator ecosystem as is. Choose it in these situations.

• When the organization already uses Airflow, so migration and consistency matter.
• When you need complex dependency orchestration that includes hybrid/multi-cloud or on-premises systems.
• When you need the rich community operators and scheduling features.

# Airflow DAG for MWAA (conceptual): Airflow orchestrates SageMaker training
from airflow import DAG
from airflow.providers.amazon.aws.operators.sagemaker import (
    SageMakerTrainingOperator,
)
 
with DAG("ml_retrain", schedule_interval="@daily") as dag:
    train = SageMakerTrainingOperator(
        task_id="train_model",
        config=training_job_config,
    )

• The trade-off is operational burden. The Airflow environment itself (scheduler and workers) stays up all the time, so it has more idle cost and more things to manage than the serverless Step Functions.
• Choosing MWAA for "a new, SageMaker-centered project" is almost always the wrong answer.

Choosing an orchestrator is a question of "how much you bundle + what existing assets you have." SageMaker jobs only → Pipelines. Serverless workflow across multiple AWS services → Step Functions. Existing Airflow assets or hybrid → MWAA. Simple trigger/schedule → EventBridge. Infrastructure definition → CloudFormation/CDK. The exam separates these five by scenario.

The Five Tools at a Glance

⚠️ Trap: An answer choice that presents EventBridge as an orchestrator is wrong. EventBridge only decides "when to start" and knows nothing about "what to do in what order." If branching, retry, or dependencies are in the requirement, the answer is Step Functions or Pipelines, and EventBridge is just the trigger attached in front of them.

When Orchestration Goes Wrong: Symptom → Cause → Action

The five tools are not competitors but layers. From the bottom they stack as IaC (what exists) → workflow (what to do in what order) → trigger (when it starts). When reading a scenario, first ask "which layer is this about?" and some answer choices are filtered out automatically.

Tomorrow we review this week's MLOps pieces by tying them into a single chain.

📖 Glossary

• IaC (Infrastructure as Code): A way of writing infrastructure such as servers, roles, and endpoints as code rather than clicks, and stamping it out as written.
• CloudFormation: AWS's basic IaC service that builds exactly what you describe when you write "I'd like it to be in this state" in YAML/JSON.
• AWS CDK: A tool that lets you write the same infrastructure as Python or TypeScript code. Synthesizing it produces a CloudFormation template.
• Stack: The unit of resource bundle that CloudFormation creates, deletes, and rolls back as one piece.
• Drift: The phenomenon where the state written in code and the actual infrastructure gradually diverge. Manual changes are the main culprit.
• Idempotency: The property that applying the same thing multiple times gives the same result as applying it once.
• Step Functions: A workflow service that strings together multiple AWS service calls as one state each and can express branching and retry. Its definition is called a state machine.
• .sync integration: A mode in which Step Functions waits until the job it invoked finishes before moving on to the next.
• EventBridge: A router that receives events and passes them to a designated target when they match a rule. It acts as the trigger of workflows.
• MWAA: Apache Airflow operated by AWS on your behalf. The option for continuing when you already have Airflow DAG assets.