Day 4 - JumpStart·Pre-trained Models·Transfer Learning + Training Cost Optimization

📌 Key Takeaways

• JumpStart is a hub of pre-trained models. There are two ways to use it — deploy as is, or fit → deploy with your own data (fine-tuning).
• Transfer learning inherits the general features of a model trained at large scale to achieve high performance with little data, short time, and low cost.
• "I have little labeled data but need an accurate classifier" → the answer is not training from scratch but transfer learning (fine-tuning).
• Training cost is split along two axes — lowering the unit price (Spot + checkpoints, appropriate instances, input mode/RecordIO) and reducing the amount of work (transfer learning, early stopping).
• The two most powerful means, Spot and transfer learning, are on different axes, so they can be applied together.

JumpStart: A Hub of Pre-trained Models

Training an image classifier or a text generation model from scratch every time costs enormous amounts of data, time, and money. JumpStart is a hub collecting hundreds of pre-trained models and solution templates, where you can deploy proven models for image classification, object detection, text classification, summarization, text generation (including LLMs), and more in a few lines, or fine-tune them with your own data.

from sagemaker.jumpstart.model import JumpStartModel
 
# Deploy the pre-trained model as is
model = JumpStartModel(model_id='huggingface-text2text-flan-t5-base')
predictor = model.deploy()
 
response = predictor.predict({'inputs': 'Summarize this sentence: ...'})

from sagemaker.jumpstart.estimator import JumpStartEstimator
 
# Fine-tune the pre-trained model with my data
estimator = JumpStartEstimator(model_id='huggingface-text2text-flan-t5-base')
estimator.fit({'training': 's3://my-bucket/my-data/'})
fine_tuned = estimator.deploy()

JumpStart splits into two usage modes.

On the exam, "quickly start with a proven model with almost no code" calls to mind JumpStart. If "specialize with my domain data" is added, it is fit → deploy.

The value of JumpStart lies in "reuse of representation learning". A model trained on large-scale data already knows general features (edges and textures in images, grammar and meaning in language). This knowledge is commonly useful for most downstream tasks. So instead of making it learn from scratch, bringing in a model that already knows is the key reason you get high performance even with little data and cost.

Transfer Learning

Transfer learning is a technique that carries knowledge learned in one task over to another related task. JumpStart fine-tuning is exactly an implementation of transfer learning. Typically, the front part of the pre-trained model (the general feature extractor) is left as is, and only the back part (the task-specific classification layer) is retrained with my data.

[Pre-trained model]  already trained on large-scale data
   Lower layers  : general features like lines·edges·textures   → freeze
   Middle layers : partial shapes·structures                    → freeze or partially unfreeze
   Output layer  : classes of the original task                 → cut off and replace
                                                    │
                                                    ▼
[My task]  new output layer matching my number of classes ── train with my data
                                                    │
                                                    ▼
                             High performance with little data and short time

There are three key benefits.

• Little data is enough: Training from scratch may need hundreds of thousands of images, but transfer learning is fine with hundreds to thousands.
• Saves training time and cost: Only some layers are trained, so it is fast and cheap.
• High performance: It inherits the knowledge of large-scale pre-training and works well even on small datasets.

How deeply to fine-tune splits on two factors: the amount of my data and domain similarity.

The reason for freezing the lower layers is that the knowledge those parts learn is reusable regardless of the task. Edges and textures in images and grammatical structure in language are needed in any task. Conversely, the output layer is tied directly to "the answer list of the original task" and cannot be reused, so it is cut off and a new one is attached. The less data you have, the more you must narrow the part you train anew to avoid overfitting.

Laying the three approaches out by cost and data requirements makes the selection criteria clear.

⚠️ Pitfall: The answer to the scenario "I have little labeled training data (e.g., hundreds of images per class) but need an accurate image classifier" is transfer learning (fine-tuning a pre-trained model). Training a CNN from scratch would leave you far short of data. Conversely, if "data is very abundant and the domain is completely different from existing models", consider training from scratch or deeper fine-tuning. The amount of data and domain similarity are the decision criteria.

Training Cost Optimization Summary

Here we gather in one place the cost-saving means that appeared scattered through this week. The exam asks comprehensively about "ways to reduce training cost".

Training cost  =  (resource unit price)  ×  (amount of work to do)
                       │                          │
                       │                          └─ Reduce the amount of work
                       │                               ├─ Transfer learning / JumpStart  (don't learn from scratch)
                       │                               ├─ Early stopping (AMT)           (stop hopeless trials)
                       │                               └─ max_run ceiling                (block runaway training)
                       │
                       └─ Lower the unit price
                            ├─ Managed Spot Training (+ checkpoints)
                            ├─ Instances matched to the algorithm (CPU / GPU)
                            ├─ Input mode (Pipe / Fast File)
                            └─ RecordIO-protobuf format

The two most powerful means are Spot training (reducing infrastructure unit price) and transfer learning/JumpStart (reducing the amount of training itself). These two are on different axes, so they can be used together — fine-tuning a pre-trained model on Spot instances stacks both savings.

# Example of cost optimization combining transfer learning + Spot
estimator = JumpStartEstimator(
    model_id='huggingface-text2text-flan-t5-base',
    use_spot_instances=True,
    max_run=3600,
    max_wait=7200,
    checkpoint_s3_uri='s3://my-bucket/checkpoints/',
)

⚠️ Pitfall: In "How do you reduce cost?", options to switch to a smaller instance are often mixed in. But if training takes proportionally longer, the total cost stays about the same or can even rise. Cost is unit price × time, so choosing by unit price alone is wrong. Conversely, moving up to a bigger GPU while GPU utilization is low is another classic wrong answer that spends money without shortening time.

Cost optimization becomes clear if you think of it in two branches. ① Lower the unit price: do the same work with cheaper resources — Spot, appropriate instance size, input mode. ② Reduce the amount of work: compute less in the first place — transfer learning (don't learn from scratch), early stopping (stop hopeless trials). If you distinguish whether an exam question is asking about "unit price" or "amount of work", the answer shows up quickly.

Additional Cost Perspective: Monitoring and Automatic Shutdown

Surprisingly common waste in training cost is "resources left on after finishing" and "training that runs endlessly because of a wrong configuration". There are several safeguards.

• Set max_run: Force-terminates training that exceeds a certain time to block runaway cost.
• Automatic notebook shutdown: Idle auto-shutdown (lifecycle config) for SageMaker Studio/notebook instances prevents billing for notebooks left on and forgotten.
• Automatic instance shutdown for training jobs: A Training Job takes its instances down automatically when finished (the flow we saw in Day 1), so unlike an endpoint the risk of billing from neglect is small.

Long-running training jobs can be checked periodically with code.

import boto3
from datetime import datetime, timedelta, timezone
 
sm = boto3.client('sagemaker')
 
# Training jobs in progress for more than 24 hours = runaway cost candidates
resp = sm.list_training_jobs(StatusEquals='InProgress', MaxResults=100)
cutoff = datetime.now(timezone.utc) - timedelta(hours=24)
 
for job in resp['TrainingJobSummaries']:
    if job['CreationTime'] < cutoff:
        print('Long-running:', job['TrainingJobName'], job['CreationTime'])

One team launched a GPU notebook instance to experiment, forgot to turn it off on Friday evening, and left it running all weekend. A training job shuts down automatically when it finishes, but a notebook instance keeps billing until a person turns it off, so a single lapse burned hundreds of thousands of Korean won. Afterward, they applied an idle auto-shutdown lifecycle config so that the notebook turns itself off after a period of inactivity, preventing a recurrence.

When Cost and Performance Diverge: Symptom → Cause → Action

In the next post, we do a comprehensive review of this whole week (Model Development 1 — SageMaker Training).

📖 Glossary

• JumpStart: SageMaker's model hub collecting proven pre-trained models and solution templates.
• Pre-trained model: A model that has already finished training on large-scale data. Use it as is or train it a bit more.
• Fine-tuning: Training a pre-trained model a bit more with my data to tailor it to my task.
• Transfer learning: A technique that carries knowledge learned in one task over to another related task. Fine-tuning is its representative implementation.
• Freeze: Removing some layers of a pre-trained model from training so their values stay fixed and do not change.
• Feature extractor / output layer: The feature extractor is the front part of the model that pulls general features (edges, textures, grammatical structure, etc.) from the input, and the output layer is the last part that turns those features into actual answer classes. When the task changes, the output layer is what gets replaced.
• Data augmentation: A technique that increases training data by creating variants of the same data through rotation, cropping, adding noise, etc.
• lifecycle config: A configuration script that runs automatically when a SageMaker notebook or Studio starts or becomes idle.
• Idle auto-shutdown: A setting that makes a notebook instance turn itself off after a period of no use. Prevents billing from neglect.
• max_run: The maximum time a training job can run. If exceeded, it is force-terminated, blocking runaway cost.