Day 2 - Network Isolation: VPC-Mode SageMaker and PrivateLink

📌 Key Takeaways

• By default, SageMaker runs containers on an AWS-managed network and can reach the internet. In regulated environments you have to block that.
• VPC mode (VpcConfig) attaches the container's ENI to a subnet in your VPC so that security groups, routing, and endpoint controls apply.
• EnableNetworkIsolation is a step stronger, cutting off all outbound calls from the container.
• To reach AWS services from a VPC with no internet gateway, you need VPC endpoints. S3 and DynamoDB use gateway endpoints; the rest use interface endpoints (PrivateLink).
• If VPC-mode training stalls, it is usually a missing S3/ECR endpoint; if distributed training doesn't work, it is a missing security-group self-referencing rule.

Three Levels of Isolation — What to Block, and How Far

If yesterday was permissions — "who can do what" — today is the network — "where does traffic flow." In the default setup, a SageMaker training job runs on an isolated network managed by AWS and can reach the internet if needed. It's convenient, but there are two stumbling blocks — ① the worry that data could leak over an internet path, and ② the financial and healthcare regulation requirement that "all traffic stay inside our VPC."

The three levels are not exclusive; they stack. Use VPC mode to pull traffic inside your own boundary, remove the internet gateway, fill the missing paths with VPC endpoints, and layer network isolation on the most sensitive workloads.

[Default mode]
Training container → AWS-managed network → (internet possible)
 
[VPC mode]
Training container → customer VPC subnet (ENI) → security group → VPC endpoint → S3 / SageMaker API
                                                  (no internet gateway)
 
[VPC mode + network isolation]
Training container ─╳─ all outbound blocked
        ▲
        └── only uses data that SageMaker pre-downloaded and mounted

💡 Related theory: The essence of VPC mode is "pulling the traffic path inside a boundary the customer can control." Once inside the VPC, you can apply all the familiar VPC controls — security groups, NACLs, route tables, VPC endpoints. Isolation begins not with "blocking" but with "gathering traffic where I can see it."

VPC Mode Configuration

You specify subnets and security groups on a training job or model with VpcConfig. SageMaker creates an ENI (Elastic Network Interface) in the specified subnets to connect the container to the VPC.

{
  "VpcConfig": {
    "Subnets": ["subnet-0a1b2c3d", "subnet-4e5f6a7b"],
    "SecurityGroupIds": ["sg-0123456789abcdef0"]
  }
}

• Subnets: Providing subnets in multiple AZs raises availability. You normally use private subnets (with no route to an internet gateway).
• SecurityGroupIds: The security groups attached to the container ENI. They control inbound and outbound rules.
• The subnet needs plenty of available IPs. Each instance needs an ENI, so in distributed training the job fails if IPs run out.

In the Python SDK, you pass them straight through as Estimator parameters.

from sagemaker.estimator import Estimator
 
estimator = Estimator(
    image_uri=container,
    role=role,
    instance_count=2,
    instance_type='ml.m5.xlarge',
    subnets=['subnet-0a1b2c3d', 'subnet-4e5f6a7b'],   # VPC mode
    security_group_ids=['sg-0123456789abcdef0'],
    enable_network_isolation=True,                     # block all outbound
    encrypt_inter_container_traffic=True,              # encrypt traffic between nodes
)

Blocking the Internet — EnableNetworkIsolation

VPC mode alone may not be enough. The stronger isolation is network isolation. When you turn it on, the container cannot make any outbound network call — not just the internet, but calls inside the VPC are blocked too. The container operates only on data provided by SageMaker.

{
  "EnableNetworkIsolation": true
}

With network isolation on, the container can't download external packages or call external APIs. So you must bake all dependencies into the container image in advance. Use it for sensitive workloads that need the strictest isolation.

EnableNetworkIsolation and VPC mode are easy to confuse but are at different levels. VPC mode is "send traffic to the customer VPC," and network isolation is "cut off all outbound from the container." Using both together is the most powerful — however, when isolation is on, SageMaker pre-downloads the training data and mounts it into the container. The answer to the scenario "prevent the container from downloading models from the internet" is network isolation.

⚠️ Pitfall: With network isolation on, pip install inside the training script, pretrained-weight downloads, and sending to an external metrics server all fail. A good share of "why does it work locally but not only on SageMaker" is this case. The responses are ① bake dependencies into a custom image, or ② upload a requirements.txt with the source directory so SageMaker handles it before startup.

VPC Endpoints (PrivateLink) — Calling AWS Services Without the Internet

If you remove the internet gateway with VPC mode, how does the container reach S3 or the SageMaker API? The answer is VPC endpoints. Traffic flows only inside the AWS backbone without passing through the internet. There are two kinds, and telling which is which is within the exam scope.

To run VPC-mode training with no internet, you need the following endpoints.

Required VPC endpoints (for VPC-mode training with no internet)
- com.amazonaws.<region>.s3                  (gateway)   → data · artifacts · ECR image layers
- com.amazonaws.<region>.sagemaker.api       (interface) → job control
- com.amazonaws.<region>.sagemaker.runtime   (interface) → inference calls
- com.amazonaws.<region>.ecr.api             (interface) → image metadata · authentication
- com.amazonaws.<region>.ecr.dkr             (interface) → Docker pull
- com.amazonaws.<region>.logs                (interface) → CloudWatch Logs
- com.amazonaws.<region>.sts                 (interface) → role assumption

If these endpoints are missing, you get the phenomenon "I switched to VPC mode and the job hangs and never finishes" — because the container can't reach S3 or ECR. In particular, ECR image layers are stored in S3, so if you create only the ECR endpoints and miss the S3 gateway endpoint, the image pull fails.

You can check what is actually created directly with the EC2 API.

import boto3
 
ec2 = boto3.client('ec2')
resp = ec2.describe_vpc_endpoints(
    Filters=[{'Name': 'vpc-id', 'Values': ['vpc-0abc123456789def0']}]
)
 
for ep in resp['VpcEndpoints']:
    print(ep['ServiceName'], ep['VpcEndpointType'], ep['State'])
    # Check that ServiceName includes all of s3 / sagemaker.api / ecr.dkr / logs / sts

The core value of PrivateLink (interface endpoints) is "AWS service traffic never rides the public internet." Since the endpoint has a private IP inside the VPC, from the container's point of view S3 or SageMaker looks as if it were in the same VPC. Moving the data-exfiltration path from the internet to the AWS backbone is the last piece of the isolation puzzle.

Where Does Traffic Actually Flow

┌──────────────────────── Customer VPC ────────────────────────┐
│                                                          │
│   Private subnet A               Private subnet B        │
│   ┌─────────────────┐           ┌─────────────────┐      │
│   │ Training container│◀─────────▶│ Training container│      │
│   │   ENI (sg-app)  │  node-to-  │   ENI (sg-app)  │      │
│   └────────┬────────┘  node      └────────┬────────┘      │
│            │  443/HTTPS                   │               │
│            ▼                              ▼               │
│   ┌────────────────────────────────────────────┐          │
│   │  Interface endpoint ENI (sg-endpoint)       │          │
│   └────────────────────────────────────────────┘          │
│            │                                              │
│            ├──▶ SageMaker API / Runtime                   │
│            ├──▶ ECR (api · dkr)                           │
│            ├──▶ CloudWatch Logs                           │
│            └──▶ STS                                       │
│                                                          │
│   Route table ──▶ [S3 gateway endpoint] ──▶ S3       │
│                                                          │
│   ✗ No internet gateway      ✗ No NAT gateway       │
└──────────────────────────────────────────────────────────┘

The traffic to the interface endpoint is made by private DNS. The container calls sagemaker.<region>.amazonaws.com as usual, and the VPC's internal DNS resolves that name to the private IP of the endpoint ENI. If you leave private DNS off, the code tries to go out to a public IP and fails, so it's effectively a required setting.

Security Groups — A Firewall at the ENI Level

You narrow traffic further with security groups attached to the container ENI and the VPC endpoint ENI. In distributed training the containers must talk to each other, so a self-referencing rule is needed.

SageMaker container security group (sg-app)
- Inbound: all TCP from sg-app itself   (node-to-node communication in distributed training)
- Outbound: 443 → sg-endpoint                (call VPC endpoints over HTTPS)
 
VPC endpoint security group (sg-endpoint)
- Inbound: 443 ← sg-app                       (HTTPS from containers)

If you allow outbound only on 443 to the endpoint security group, traffic doesn't leak elsewhere. The trick is to reference other security group IDs rather than IP ranges. In a SageMaker environment where ENIs get a new IP each time, IP-based rules are impossible to manage.

VPC Mode Checklist

The order of checks when VPC-mode training stalls or fails:

1. Is the subnet private, and does routing go to the VPC endpoints?
2. Does the S3 gateway endpoint exist and is it associated with that subnet's route table?
3. Do the SageMaker API/Runtime, ECR (api, dkr), Logs, and STS interface endpoints exist?
4. Is private DNS on for the interface endpoints?
5. Is security group outbound 443 open to the endpoint SG?
6. For distributed training, is there a self-referencing inbound rule?
7. Does the subnet have available IPs left to create ENIs?
8. Is external package downloading truly necessary (if so, a NAT gateway or baking into the image)?

When Isolation Gets Blocked: Symptom → Cause → Action

Tomorrow we cover KMS encryption and Secrets management, which protect the data and the model themselves.

📖 Glossary

• VPC mode: A setting that runs training/inference containers attached as ENIs to customer VPC subnets. Specified with Subnets and SecurityGroupIds.
• ENI (Elastic Network Interface): A virtual network card with a private IP inside a VPC. SageMaker creates it when attaching a container to the VPC.
• EnableNetworkIsolation: An option that blocks all outbound network calls from the container. The strongest means of isolation.
• VPC endpoint: A pathway that lets you reach AWS services without passing through the internet. There are gateway and interface types.
• Gateway endpoint: Dedicated to S3 and DynamoDB. It works by adding a route to the route table and has no extra cost.
• Interface endpoint (PrivateLink): Creates an ENI and a private IP in the subnet to connect to the service. Most services use this method, billed per hour and per data.
• Private DNS: A feature that resolves a service's public domain name to the endpoint's private IP. If left off, code tries to go out over a public path and fails.
• Private subnet: A subnet with no route to an internet gateway. The default placement for VPC-mode SageMaker.
• Self-referencing security group rule: An inbound rule that specifies the same security group as its source. Essential for node-to-node communication in distributed training.
• aws:SourceVpce: A condition key that checks whether a request came in through a specific VPC endpoint. Used when opening a bucket only through that path.