← Back to Table of Contents

Chapter 19 β€” Detection Architectures

β€œDetection architectures encode years of hard-won engineering intuition: how to handle scale, localize without anchors, and match the right proposal to the right ground truth.”

The R-CNN Family: Two-Stage Pioneers

Planned content: R-CNN (2013): selective search proposals β†’ warp β†’ CNN β†’ SVM classifier. Slow (2000 crops per image). Fast R-CNN: single forward pass, RoIPool for feature extraction. Faster R-CNN: replace selective search with Region Proposal Network (RPN). RPN shares backbone features. Cascade R-CNN: multi-stage bounding box refinement with increasing IoU thresholds.

πŸ“Š Planned diagram: R-CNN family evolution β€” R-CNN vs. Fast R-CNN vs. Faster R-CNN, showing computation flow and speed improvement at each step.

πŸ“Š Planned diagram: Faster R-CNN architecture β€” backbone β†’ FPN β†’ RPN (objectness + box) β†’ RoIAlign β†’ classification + regression heads. Tensor shapes throughout.

SSD and RetinaNet

Planned content: SSD: multi-scale anchors from multiple feature map levels. RetinaNet: FPN + focal loss + class/box subnets. Why focal loss enabled single-stage to match two-stage accuracy. RetinaNet as the canonical one-stage baseline.

πŸ“Š Planned diagram: RetinaNet with FPN β€” P3 to P7 feature maps, shared class and box subnets, anchor grid visualization.

YOLO Series: Speed-Accuracy Pareto Frontier

Planned content: YOLO (v1–v11) is the most deployed detection family. Each version made key architectural or training improvements.

YOLOv1 (2015)

Planned content: Divide image into SΓ—S grid. Each cell predicts B boxes + confidence + C class probs. Single regression output. 45 FPS. The unified detection paradigm.

πŸ“Š Planned diagram: YOLOv1 grid prediction β€” [S, S, B*5+C] output tensor explained cell by cell.

YOLOv2 (2016) β€” YOLO9000

Planned content: Anchor boxes, batch normalization, high-resolution classifier, multi-scale training. WordTree for 9000-class detection.

YOLOv3 (2018)

Planned content: Multi-scale prediction (3 scales, like FPN). Darknet-53 backbone. Logistic classifier instead of softmax (multi-label). 320-608 input sizes. Still widely used baseline.

πŸ“Š Planned diagram: YOLOv3 multi-scale output β€” three detection heads at strides 8, 16, 32, each predicting [B, 3*(5+C), H/s, W/s].

YOLOv4 (2020)

Planned content: Bag of Freebies (training tricks) + Bag of Specials (architectural tricks). CSPNet backbone, PANet neck, Mish activation, mosaic augmentation, DropBlock.

YOLOv5

Planned content: Ultralytics implementation. YAML-configurable. P2/P3/P4/P5 model variants. Widely adopted in production. AutoAnchor. FP16 export.

YOLOv6, YOLOv7, YOLOv8

Planned content: YOLOv6 (Meituan): RepVGG blocks, decoupled head. YOLOv7: E-ELAN, COCO SOTA in 2022. YOLOv8 (Ultralytics): anchor-free, unified interface for detection/segmentation/pose/classification.

πŸ“Š Planned diagram: YOLOv8 architecture β€” CSPDarknet backbone β†’ C2f blocks β†’ FPN+PAN neck β†’ decoupled anchor-free head with DFL (Distribution Focal Loss).

YOLOv9, YOLOv10, YOLOv11

Planned content: YOLOv9: GELAN + PGI (Programmable Gradient Information). YOLOv10: NMS-free end-to-end detection. YOLOv11: Ultralytics v11 with C3k2 blocks and efficiency improvements.

πŸ“Š Planned table: YOLO version comparison β€” year, backbone, neck, head type, anchor-free?, COCO mAP, inference speed.

Anchor-Free Methods

Planned content: FCOS: per-pixel box regression (l, r, t, b) + centerness score. CenterNet: detect objects as center points on heatmap + offset + size. CornerNet: detect corners via corner pooling + associative embedding.

πŸ“Š Planned diagram: CenterNet head β€” three branches: heatmap [B, C, H/4, W/4] + offset [B, 2, H/4, W/4] + size [B, 2, H/4, W/4]. Decoding process.

DETR and Transformer-Based Detection

Planned content: DETR: remove anchors and NMS entirely. N object queries β†’ cross-attention with image features β†’ directly predict N boxes and classes. Bipartite matching loss (Hungarian). Slow convergence. Deformable DETR: deformable attention for faster training. DN-DETR, DAB-DETR, DINO (Detection Transformers).

πŸ“Š Planned diagram: DETR architecture β€” backbone β†’ positional encoding β†’ encoder-decoder transformer β†’ N queries β†’ prediction heads β†’ Hungarian matching with GT.

πŸ“Š Planned diagram: Hungarian bipartite matching in DETR training β€” N predictions matched to M GT boxes (NΒ Β» M), showing the assignment and unmatched queries predicting β€œno object”.

RT-DETR: Real-Time Transformer Detection

Planned content: PaddleDetection’s RT-DETR: hybrid encoder (CNN + attention), efficient decoder. Matches YOLO speed with DETR accuracy. RT-DETRv2.

πŸ“Š Planned diagram: RT-DETR vs. YOLO on accuracy-speed scatter plot.

Architecture Comparison

πŸ“Š Planned table: Detection architectures β€” anchor type, backbone, neck, NMS required, COCO AP, latency (ms) on T4.

Next: Chapter 20 β€” Segmentation Architectures β†’


Last updated: May 2026