← Back to Table of Contents

Chapter 18 β€” Classification Architectures

β€œEvery architecture is a hypothesis about what matters in images β€” depth, skip connections, scale, or attention.”

AlexNet and the Deep Learning Revolution (2012)

Planned content: 5 conv layers + 3 FC layers, ReLU, dropout, data augmentation, GPU training. Why it was revolutionary: 10.8% top-5 error vs. 25.8% for runner-up on ImageNet 2012. Local response normalization. Landmark paper anatomy.

πŸ“Š Planned diagram: AlexNet architecture with tensor shapes β€” [B, 3, 227, 227] β†’ 5 conv blocks β†’ [B, 256, 6, 6] β†’ flatten β†’ 3 FC β†’ [B, 1000].

VGG: Depth and Simplicity (2014)

Planned content: VGG-16, VGG-19. Only 3Γ—3 convolutions. Very deep (16-19 layers). 138M parameters β€” large by modern standards. Why uniform architecture is good for transfer learning. Still used as backbone in feature extraction.

ResNet: Residual Connections (2015)

Planned content: The degradation problem: deeper networks perform worse without skip connections (not overfitting β€” training accuracy also degrades). Solution: y = F(x) + x. Basic block (2 conv) vs. Bottleneck block (1Γ—1 β†’ 3Γ—3 β†’ 1Γ—1 for deep ResNets). ResNet-18/34/50/101/152. Pre-activation ResNet (ResNetV2). Wide ResNet. ResNeXt: grouped convolutions.

πŸ“Š Planned diagram: ResNet basic block and bottleneck block β€” tensor shapes [B, 64, 56, 56] β†’ block β†’ [B, 64, 56, 56] with shortcut connection.

πŸ“Š Planned diagram: ResNet architecture pyramid β€” stage 1 (64ch, stride 4) β†’ stage 2 (128ch, stride 2) β†’ stage 3 (256ch) β†’ stage 4 (512ch) β†’ GAP β†’ FC.

MobileNet: Efficiency (v1/v2/v3)

Planned content: MobileNetV1: depthwise separable convolutions throughout. MobileNetV2: inverted residuals + linear bottlenecks. Why linear activation in bottleneck prevents information loss. ReLU6. MobileNetV3: NAS-designed, hard-swish, squeeze-and-excite.

πŸ“Š Planned diagram: Inverted residual block β€” narrow β†’ wide β†’ narrow (reversed from ResNet), with linear bottleneck and SE block.

EfficientNet: Compound Scaling (2019)

Planned content: Compound scaling: simultaneously scale depth, width, and resolution using a fixed coefficient Ο†. NAS to find the baseline architecture. EfficientNet-B0 to B7. EfficientNetV2: progressive training, smaller early-stage conv. Pareto frontier vs. ResNet.

πŸ“Š Planned diagram: Compound scaling illustration β€” depth Ξ±^Ο†, width Ξ²^Ο†, resolution Ξ³^Ο† scaling from B0.

πŸ“Š Planned diagram: EfficientNet accuracy vs. FLOPs Pareto curve β€” comparing with ResNet, MobileNet, NASNet.

Vision Transformer (ViT): Patches as Tokens (2020)

Planned content: Divide image into 16Γ—16 patches β†’ flatten β†’ linear projection to d_model. Add positional embeddings. Prepend CLS token. Pass through standard Transformer encoder (L layers). CLS token β†’ classification head. Requires large data (ViT-L trained on JFT-300M). DeiT: train ViT on ImageNet with distillation.

πŸ“Š Planned diagram: ViT architecture β€” [B, 3, 224, 224] β†’ patch embedding [B, 196, 768] β†’ add CLS token [B, 197, 768] β†’ L Γ— Transformer block β†’ CLS token β†’ head [B, 1000].

1
2
3
4
5
6
7
8
9
10
11
# ViT patch embedding
B, C, H, W = 8, 3, 224, 224
patch_size = 16
n_patches = (H // patch_size) * (W // patch_size)  # 196

patch_embed = nn.Conv2d(C, 768, kernel_size=patch_size, stride=patch_size)
# Equivalent to flattening 16Γ—16 patches and projecting

x = torch.randn(B, C, H, W)         # [8, 3, 224, 224]
x = patch_embed(x)                   # [8, 768, 14, 14]
x = x.flatten(2).transpose(1, 2)    # [8, 196, 768]

Swin Transformer: Hierarchical Windows (2021)

Planned content: Key insight: ViT global attention is O(nΒ²) β€” impractical for dense prediction. Swin: local window attention (7Γ—7 patches per window) β€” O(n). Shifted windows between layers for cross-window communication. Hierarchical feature maps (like CNN). Swin-T/S/B/L variants. Universal backbone for detection, segmentation, classification.

πŸ“Š Planned diagram: Swin shifted window attention β€” two consecutive layers, showing W-MSA (regular windows) and SW-MSA (shifted windows).

πŸ“Š Planned diagram: Swin vs. ViT feature hierarchy β€” Swin produces [H/4, W/4], [H/8, W/8], [H/16, W/16], [H/32, W/32] feature maps for FPN.

ConvNeXt: Modernizing CNNs (2022)

Planned content: Take ResNet-50 and add every Swin/ViT design choice one by one. Result: ConvNeXt block β€” large kernel (7Γ—7) depthwise conv + inverted bottleneck + LayerNorm + GELU + fewer activation/norm layers. Matches or beats Swin on all benchmarks. Strong argument for CNNs with modern training.

πŸ“Š Planned diagram: ConvNeXt block vs. ResNet bottleneck β€” side-by-side showing the modernization changes.

Architecture Comparison Table

Planned content: Comprehensive comparison table of all major classification architectures.

πŸ“Š Planned table: Architecture, year, ImageNet top-1 %, parameters (M), FLOPs (G), inference latency (ms on V100), training data needed.

Next: Chapter 19 β€” Detection Architectures β†’


Last updated: May 2026