← Back to Table of Contents

Chapter 21 β€” Pose Estimation Architectures

β€œThe architecture for pose must preserve high-resolution spatial detail while capturing global context β€” a tension that defines the field.”

Stacked Hourglass Networks (2016)

Planned content: Stack multiple encoder-decoder β€œhourglass” modules sequentially. Each hourglass: downsample to global context β†’ upsample to full resolution with skip connections. Stack 4-8 hourglasses with intermediate supervision at each. Bottom-up processing + top-down refinement iteratively.

πŸ“Š Planned diagram: Stacked Hourglass β€” two chained hourglass modules with intermediate heatmap supervision, showing tensor flow [B, K, 64, 64] at each stage.

SimpleBaseline (2018)

Planned content: Deceptively simple: ResNet backbone + 3 deconvolutional (transposed conv) layers. Works surprisingly well. Lightweight, easy to train. Established strong baseline for COCO top-down.

πŸ“Š Planned diagram: SimpleBaseline β€” ResNet β†’ 3 deconv layers [B, 256, 64, 64] β†’ 1Γ—1 conv β†’ heatmaps [B, K, 64, 64].

HRNet: High-Resolution Net (2019)

Planned content: Key insight: don’t sacrifice spatial resolution. Maintain high-resolution representations throughout the network. Multi-resolution parallel branches: HRNet simultaneously processes [B, C, H/4, W/4], [B, 2C, H/8, W/8], [B, 4C, H/16, W/16], [B, 8C, H/32, W/32]. Multi-resolution fusion: exchange information between branches at each stage. Final heatmap predicted from high-resolution branch.

πŸ“Š Planned diagram: HRNet multi-resolution parallel branches β€” 4 resolution levels maintained in parallel, with fusion modules at each stage. Contrast with downsampled-then-upsampled alternatives.

πŸ“Š Planned diagram: HRNet fusion block β€” bilinear upsample of lower-res branches + strided conv downsample of higher-res branches β†’ add β†’ fused feature maps.

πŸ“Š Planned table: HRNet-W32 vs. HRNet-W48 β€” parameters, GFLOPs, COCO keypoint AP.

ViTPose: Vision Transformer for Pose (2022)

Planned content: Plain ViT (non-hierarchical) as backbone for pose. Simple decoder (2 deconv layers). Scale up to ViT-H for SOTA. Multi-task training on multiple pose datasets. Why ViT representations transfer well to pose estimation.

πŸ“Š Planned diagram: ViTPose β€” ViT-B/L/H patch tokens [B, 256, 768] β†’ reshape to [B, 768, 16, 16] β†’ deconv decoder β†’ heatmaps [B, K, 64, 64].

OpenPose: Bottom-Up with PAFs (2017)

Planned content: (PAFs covered in depth in Ch. 7) β€” Architecture: VGG-based two-branch network predicting heatmaps + PAFs simultaneously. Multi-stage refinement (6 stages). COCO whole-body extension. Widely deployed real-time pose.

πŸ“Š Planned diagram: OpenPose architecture β€” shared VGG features β†’ two branches (heatmap, PAF) Γ— 6 stages β†’ assembly.

DEKR: Disentangled Keypoint Regression (2021)

Planned content: Bottom-up anchor-free approach. Adaptive convolution centered on each pixel. Disentangled learning: separate feature extraction per keypoint type. Eliminates heatmap β†’ coordinate quantization error. HRNet backbone.

WholeBody Pose

Planned content: ZoomNet: zoom into face/hand/feet regions for fine-grained detection. OSX: one-stage expressive whole-body. ExPose: part-specific feature extractors. 133-keypoint COCO WholeBody benchmark.

Next: Chapter 22 β€” Face Architectures β†’


Last updated: May 2026