Day 2 - Built-in Algorithms: XGBoost, Linear Learner, Image·Text, Input Formats

📌 Key Takeaways

• Built-in algorithms are containers AWS has prebuilt. You do not write training code — if you give the data in the correct format, training works.
• The heart of the exam is the problem type → algorithm mapping — tabular classification/regression = XGBoost, large-scale linear/sparse = Linear Learner, anomaly detection = RCF, time series = DeepAR, sparse recommendation = Factorization Machines, fast text = BlazingText.
• Image algorithms split into three branches: classification (one label) / detection (location + class) / segmentation (pixel level).
• Input formats: large-volume streaming uses RecordIO-protobuf; small and simple uses CSV.
• XGBoost's CSV rule — the first column is the target (label), and there is no header row. This is a rule you need to know for the exam.

Why Use Built-in Algorithms

It is a choice between writing PyTorch yourself and using an algorithm AWS has optimized. The built-in side has three advantages.

• No code needed: You do not have to write a training script.
• Optimization: AWS has pre-tuned distributed training and GPU utilization.
• Scalability: Designed for large data (especially the RecordIO format + Pipe mode).

Even for the same training, the amount of hands-on work differs depending on "what you pick up and use".

from sagemaker import image_uris
 
# Look up the built-in algorithm container URI
container = image_uris.retrieve('xgboost', region, version='1.7-1')
 
estimator = Estimator(
    image_uri=container,   # just specify the built-in container and you're done
    role=role,
    instance_count=1,
    instance_type='ml.m5.xlarge',
)

Built-in algorithms are an implementation of the principle "don't reinvent the wheel". XGBoost and logistic regression are algorithms validated over decades, and using a container where AWS has optimized distribution, streaming, and even GPU avoids the bugs and inefficiencies that arise when you write it yourself. It is reasonable to drop down to custom code only when you truly need a new architecture.

Representative Algorithms and Problem-Type Mapping

The exam asks "Which built-in algorithm is best suited to this problem?" The problem type → algorithm mapping is as follows.

Rather than memorizing everything, firmly grasp the core algorithms. Tabular classification/regression is XGBoost, large-scale linear is Linear Learner, anomaly detection is RCF, time series is DeepAR, sparse-data recommendation is Factorization Machines, and fast text is BlazingText. Knowing just these six for sure clears up a good share of the answer choices.

Following the clues in the question stem generally narrows things down to one branch.

What is the data?
├─ Table (tabular)
│   ├─ Labels (answers) available
│   │   ├─ Dense features, non-linear relationships   → XGBoost
│   │   ├─ Tens of thousands of features, sparse, fast linear   → Linear Learner
│   │   └─ Very sparse like user×item                 → Factorization Machines
│   └─ No labels
│       ├─ Group similar things together              → K-Means
│       ├─ Find rare outliers                         → Random Cut Forest
│       └─ Reduce the number of features              → PCA
├─ Time series (several series together)              → DeepAR
├─ Text
│   ├─ Embeddings·fast classification                 → BlazingText
│   └─ Extract hidden topics of documents             → LDA / NTM
└─ Image
    ├─ One label per photo                            → Image Classification
    ├─ Object location + class                        → Object Detection
    └─ A class for every pixel                        → Semantic Segmentation

XGBoost and Linear Learner

XGBoost is a gradient boosting tree algorithm and is effectively the default choice for classification and regression on structured (tabular) data. It is powerful in practice thanks to missing-value handling, capturing non-linear relationships, and providing feature importance.

estimator.set_hyperparameters(
    objective='binary:logistic',  # binary classification
    num_round=100,                # number of boosting rounds
    max_depth=5,                  # tree depth
    eta=0.2,                      # learning rate
    subsample=0.8,                # row sampling ratio
)

Linear Learner implements linear/logistic regression for large-scale data. It is simpler than XGBoost, but suits cases where the data is well separated linearly or has a very large number of features, and where fast training is needed. It also has the property of training several models in parallel and automatically selecting the best one.

The boundaries among the three structured-data algorithms are drawn by the shape of the data.

⚠️ Pitfall: For "classification of structured tabular data", the safe answer is XGBoost. However, if "there are tens of thousands of features, very sparse, and a fast linear model is needed", Linear Learner fits better. And for "very sparse recommendation like user-item", it is Factorization Machines. The trick is to separate the three by the data shape in the scenario (dense table vs. high-dimensional sparse vs. recommendation-sparse).

Image and Text Algorithms

The image family distinguishes three algorithms in the order of increasing "fineness of output".

If the question stem mentions "location (where)", Image Classification is out, and if it demands even "pixel-level, region boundaries", it is Semantic Segmentation.

For text, BlazingText is the representative. It handles Word2Vec embedding training and text classification very quickly (a highly optimized implementation). For "train word embeddings quickly" or "large-scale text classification", think of BlazingText.

# BlazingText: text classification mode
estimator.set_hyperparameters(
    mode='supervised',   # supervised=classification, skipgram/cbow=embedding
    epochs=10,
    word_ngrams=2,
)

Word2Vec (BlazingText's embedding mode) is based on the distributional hypothesis that "words that appear together have similar meanings". It learns each word as a vector, making semantic arithmetic like "king - man + woman ≈ queen" possible. These embeddings are later reused as input features for classification, search, and recommendation. Just as a CNN in images extracts visual features from pixels, embeddings are representation learning that extracts semantic features from words.

Input Formats: RecordIO-protobuf vs CSV

For built-in algorithms, the input format is within the exam scope. Distinguish two.

RecordIO-protobuf serializes data densely in binary, making it the most efficient when streaming large volumes in Pipe mode. Many built-in algorithms (especially Linear Learner, Factorization Machines, K-Means, etc.) recommend or require this format. CSV is also supported, but for large volumes RecordIO is faster.

import sagemaker.amazon.common as smac
import io
 
# Serialize data to RecordIO-protobuf and upload to S3
buf = io.BytesIO()
smac.write_numpy_to_dense_tensor(buf, features, labels)
buf.seek(0)

XGBoost is an exception, mainly using CSV and libsvm (a property of tree algorithms). The exam also asks about the rule that when giving CSV, the first column is the target (label) and there must be no header.

The format and input mode are specified together when creating the channel.

from sagemaker.inputs import TrainingInput
 
# XGBoost + CSV (first column is the label, no header)
csv_input = TrainingInput(
    's3://my-bucket/train/train.csv',
    content_type='text/csv',
)
 
# Large volume: RecordIO-protobuf + Pipe mode
pipe_input = TrainingInput(
    's3://my-bucket/train/',
    content_type='application/x-recordio-protobuf',
    input_mode='Pipe',
)
 
estimator.fit({'train': csv_input})

Because the expected input differs by algorithm, remember the formats of the representative algorithms as a group.

The essence of the format debate is "parsing cost". CSV writes numbers as human-readable strings, so the training instance repeatedly converts strings back to numbers. RecordIO-protobuf is a machine-readable binary representation from the start, so this conversion is absent and the size is smaller too. With small data the difference is invisible, but when streaming hundreds of GB, parsing becomes a bottleneck that leaves the GPU idle.

When You Pick the Wrong Algorithm or Format: Symptom → Cause → Action

The exam checks algorithm and format knowledge with situations like "I did this, but the result is strange".

⚠️ Pitfall: To "train efficiently by streaming large data in Pipe mode", the format is RecordIO-protobuf. CSV is text, so it is large and has parsing cost, making it inefficient for large-volume streaming. On the other hand, if asked "the label position when training XGBoost with CSV", it is the first column and there must be no header. Keep format efficiency (RecordIO) and XGBoost's CSV label rule (first column, no header) separate.

In the next post, we look at SageMaker AMT, which automatically optimizes the hyperparameters of the algorithm you chose this way.

📖 Glossary

• Built-in algorithm: A training container AWS has prebuilt. Training works by giving only data and hyperparameters, without writing code.
• XGBoost: An algorithm that chains many small decision trees to reduce error step by step. The de facto default choice for tabular data.
• Gradient boosting: A technique that stacks multiple models so that each next model focuses on correcting what the previous models got wrong.
• Linear Learner: A built-in algorithm implementing linear·logistic regression for large-scale data. Strong when there are very many features.
• Factorization Machines / sparse data: Sparse data is data where most values are 0 or blank (a user×item purchase matrix is the classic example). FM is a recommendation algorithm that learns interactions between items as latent factors from such extremely sparse data.
• Random Cut Forest (RCF): An anomaly detection algorithm that finds data that "stands out unusually" without any labels.
• DeepAR: A deep-learning-based demand/numeric forecasting algorithm that trains many time series at once and borrows patterns from one another.
• BlazingText / embedding: An embedding turns a word or item into a numeric vector, where similar meanings give nearby vectors. BlazingText is an algorithm optimized to handle this word embedding (Word2Vec) training and text classification very quickly.
• Bounding box: A rectangular region enclosing an object within an image. The output form of object detection.
• RecordIO-protobuf / libsvm: RecordIO-protobuf is a format that stores training data densely in binary, with a small size and fast parsing, favorable for large-volume streaming. libsvm is a text format for sparse data that writes only non-zero values in the form label index:value, and XGBoost supports it.