Chapter 21 β Pose Estimation Architectures
βThe architecture for pose must preserve high-resolution spatial detail while capturing global context β a tension that defines the field.β
Stacked Hourglass Networks (2016)
Planned content: Stack multiple encoder-decoder βhourglassβ modules sequentially. Each hourglass: downsample to global context β upsample to full resolution with skip connections. Stack 4-8 hourglasses with intermediate supervision at each. Bottom-up processing + top-down refinement iteratively.
π Planned diagram: Stacked Hourglass β two chained hourglass modules with intermediate heatmap supervision, showing tensor flow
[B, K, 64, 64]at each stage.
SimpleBaseline (2018)
Planned content: Deceptively simple: ResNet backbone + 3 deconvolutional (transposed conv) layers. Works surprisingly well. Lightweight, easy to train. Established strong baseline for COCO top-down.
π Planned diagram: SimpleBaseline β ResNet β 3 deconv layers
[B, 256, 64, 64]β 1Γ1 conv β heatmaps[B, K, 64, 64].
HRNet: High-Resolution Net (2019)
Planned content: Key insight: donβt sacrifice spatial resolution. Maintain high-resolution representations throughout the network. Multi-resolution parallel branches: HRNet simultaneously processes
[B, C, H/4, W/4],[B, 2C, H/8, W/8],[B, 4C, H/16, W/16],[B, 8C, H/32, W/32]. Multi-resolution fusion: exchange information between branches at each stage. Final heatmap predicted from high-resolution branch.
π Planned diagram: HRNet multi-resolution parallel branches β 4 resolution levels maintained in parallel, with fusion modules at each stage. Contrast with downsampled-then-upsampled alternatives.
π Planned diagram: HRNet fusion block β bilinear upsample of lower-res branches + strided conv downsample of higher-res branches β add β fused feature maps.
π Planned table: HRNet-W32 vs. HRNet-W48 β parameters, GFLOPs, COCO keypoint AP.
ViTPose: Vision Transformer for Pose (2022)
Planned content: Plain ViT (non-hierarchical) as backbone for pose. Simple decoder (2 deconv layers). Scale up to ViT-H for SOTA. Multi-task training on multiple pose datasets. Why ViT representations transfer well to pose estimation.
π Planned diagram: ViTPose β ViT-B/L/H patch tokens
[B, 256, 768]β reshape to[B, 768, 16, 16]β deconv decoder β heatmaps[B, K, 64, 64].
OpenPose: Bottom-Up with PAFs (2017)
Planned content: (PAFs covered in depth in Ch. 7) β Architecture: VGG-based two-branch network predicting heatmaps + PAFs simultaneously. Multi-stage refinement (6 stages). COCO whole-body extension. Widely deployed real-time pose.
π Planned diagram: OpenPose architecture β shared VGG features β two branches (heatmap, PAF) Γ 6 stages β assembly.
DEKR: Disentangled Keypoint Regression (2021)
Planned content: Bottom-up anchor-free approach. Adaptive convolution centered on each pixel. Disentangled learning: separate feature extraction per keypoint type. Eliminates heatmap β coordinate quantization error. HRNet backbone.
WholeBody Pose
Planned content: ZoomNet: zoom into face/hand/feet regions for fine-grained detection. OSX: one-stage expressive whole-body. ExPose: part-specific feature extractors. 133-keypoint COCO WholeBody benchmark.
Next: Chapter 22 β Face Architectures β
Last updated: May 2026