Day 2 - CNN: Convolutional Neural Networks and Computer Vision

📌 Key Takeaways

• A CNN processes images efficiently thanks to three ideas: local receptive fields + weight sharing + translation invariance.
• Convolution extracts patterns and pooling shrinks the size. Pooling has no learnable parameters.
• You must clearly distinguish classification (labels) / object detection (boxes + classes) / semantic segmentation (pixels).
• ResNet's residual (skip) connections are the fix for the problem that networks of 100+ layers would not train.
• SageMaker provides the Image Classification, Object Detection, and Semantic Segmentation built-ins plus Ground Truth labeling.

Why Convolution, Not Fully Connected?

Images and video have an enormous number of pixels. Feeding them straight into a fully connected (Dense) neural network makes the parameter count explode, and the network also fails to learn translation invariance. A convolutional neural network (CNN) solves this by sliding small filters across the whole image to extract local patterns.

• A 224×224×3 color image flattened into a Dense layer gives roughly 150,000 input nodes.
• If the first hidden layer has 1,000 nodes, that is 150 million weights — an explosion.

The three ideas a CNN uses:

• Local receptive field: one neuron sees only a small region, not the whole image.
• Weight sharing: the same filter is reused at every position, which collapses the parameter count.
• Translation invariance: a cat is detected with the same filter wherever it is.

Weight sharing rests on the assumption that "a filter that finds a cat's ear works exactly the same way at any position in the image." This assumption reduces the parameters to learn and lets the model generalize well even on limited data. It builds the property "the meaning of an image is preserved under spatial shifts" into the structure itself as an inductive bias.

MLS-C01 does not ask about the internal math of CNNs. It asks about the roles of convolution and pooling, image classification vs object detection, and choosing SageMaker built-ins and architectures.

Convolution Layer

A small filter (kernel) slides over the input and a dot product is computed at each position, producing a feature map.

Input image  *  Filter (e.g., 3x3)  →  Feature map
(emphasizes edges, texture, shape patterns)

Convolution Hyperparameter Table

Stride 1 + same padding           Stride 2
┌───────────┐                    ┌───────────┐
│ ▢▢▢▢▢▢▢▢ │ → size preserved    │ ▢▢▢▢▢▢▢▢ │ → shrinks to half size
└───────────┘                    └───────────┘

Abstraction deepens as layers are stacked: low level (edges) → mid level (texture, parts) → high level (whole objects).

One side of the output is determined by the four hyperparameters:

output size = floor( (input size + 2×padding − kernel size) / stride ) + 1
output channels = number of filters

As layers stack up, the region of the original image that one neuron sees (its receptive field) grows. A single 3×3 filter sees only 3×3, but stacking another 3×3 on top makes it see 5×5 relative to the original. That is why a VGG-style design that stacks small kernels deeply can see a wide region with fewer parameters than one large kernel. Pooling and large strides make this expansion even faster.

• If the question says "we want to keep the output size unchanged," the answer is same padding. Increasing the stride or choosing valid padding shrinks the size.

Pooling Layer

Pooling reduces the spatial size of the feature map, lowering computation and making the model robust to small position changes.

• Pooling has no learnable parameters (it is a fixed operation).
• Downsampling widens the receptive field and reduces overfitting.

⚠️ Trap: An option saying "pooling adds weights to raise expressiveness" is wrong. Adding weights is the role of convolution and Dense layers.

Typical CNN Architecture

[Conv → ReLU → Pool] × N  →  Flatten/GAP  →  Dense  →  Softmax
   Feature extraction (backbone)            Classification head

• The front part (conv/pool blocks) is the feature extractor (backbone).
• The back part (Dense + Softmax) is the classification head.
• Transfer learning (Day 4) often reuses the backbone and trains only the head.

• Flatten spreads the entire feature map into the Dense layer, so parameters surge. In a question saying "the fully connected layer has too many parameters," replacing it with Global Average Pooling is the right direction.

Representative Architectures

Knowing the names and core ideas gives you clues in the question stem.

There was a paradox where making a network deeper actually made training worse (vanishing gradients). ResNet's residual/skip connection adds the input directly to the output (H(x)=F(x)+x), creating a shortcut for gradients to flow, so networks of 100+ layers train stably. The exam asks about it as the fix for "a deep network fails to train."

Image Classification vs Object Detection vs Segmentation

Distinguishing the CNN applications precisely is the heart of the exam.

⚠️ Trap: "How many are where" is detection; "separate regions pixel by pixel" is segmentation. A single verb in the stem ("count/locate" vs "pixel-level") decides the answer.

SageMaker Built-ins and Data Formats

• Image Classification: single- or multi-label classification. ResNet-based. Supports a transfer learning mode.
• Object Detection: SSD-based. Outputs boxes + classes.
• Semantic Segmentation: pixel-level classification.
• Input is usually RecordIO (recommended, fast) or image files + annotations. For large data, use Pipe mode streaming.
• Labeling: SageMaker Ground Truth (provides task templates for bounding boxes and segmentation).

Situation → AWS Choice Mapping Table

When Data Is Scarce: Augmentation and Transfer Learning

When images are scarce, data augmentation (rotation, flipping, cropping, color jitter) increases training diversity and reduces overfitting. SageMaker Image Classification and Object Detection have augmentation options built in. Also, transfer learning with ImageNet-pretrained weights performs better and takes less time than training from scratch on little data (linked to Day 4).

Apply augmentation to the training set only. Augmenting the validation or test set distorts the evaluation.

Reading Classification Performance — Metric Interpretation Table

Image classification is still classification. When classes are skewed, accuracy misleads people.

Question Clue → Answer Mapping

If you see only "improve accuracy" and automatically pick a deeper architecture, you will be wrong. When there is also a clue that data is scarce, the answer is not a bigger structure but transfer learning and augmentation.

Symptom → Cause → Remedy

The next post covers data where order, not space, creates meaning — RNN, LSTM, GRU, and the Transformer.

📖 Glossary

• Convolution: an operation that slides a small filter over the input and computes dot products to extract local patterns.
• Filter (kernel) / feature map: the small grid of weights a convolution learns / the output produced by passing the input through that filter. One channel is created per filter.
• Stride / padding: the step size of the filter / a setting that pads the edges with 0s to control the output size.
• Local receptive field: the small range of input one neuron sees. It widens as layers get deeper.
• Weight sharing: reusing the same filter at every position in the image. Parameters shrink greatly.
• Translation invariance: the property that an object is still detected with the same filter even if it moves within the image.
• Pooling: an operation that shrinks the feature map using the maximum or average of a region. It has no learnable parameters.
• Residual (skip) connection: a structure that adds the input directly to the output, creating a shortcut for gradients to flow. The core of ResNet.
• Backbone / head: the front part that extracts features / the back part that produces the final prediction. Transfer learning usually reuses the backbone.
• RecordIO: the input format SageMaker recommends, which bundles many images to favor sequential reads. It is used together with the bounding-box annotations that object detection outputs.