ποΈ Computer Vision β The Complete Guide
From Pixels to Predictions: Understanding, building, and deploying ML-based computer vision systems. A comprehensive reference for practitioners, engineers, and researchers navigating the modern CV landscape.
Who This Is For
You understand Python and have touched PyTorch or TensorFlow. Youβve used a pre-trained model, maybe fine-tuned one. But you want to truly understand what is happening inside: how a convolutional filter fires, why NMS is tricky, how ArcFace creates separable embedding spaces, how YOLOβs anchor-free head works, and what makes a transformer see an image differently than a CNN. This guide takes you from first principles β images as tensors β all the way to foundation models, generative systems, and the research frontier.
π Table of Contents
| # | Chapter | What Youβll Learn |
|---|---|---|
| Overview & Foundations | Β | Β |
| 0 | Grand Overview | The full CV landscape β tasks, history, benchmarks, and how every piece connects |
| 1 | Image Fundamentals | Images as tensors [B, C, H, W], color spaces, preprocessing pipelines, camera models |
| Use Cases | Β | Β |
| 2 | Image Classification | Softmax, cross-entropy, fine-grained recognition, zero-shot classification, ImageNet |
| 3 | Object Detection | Two-stage vs one-stage, anchors, NMS deep dive, IoU variants, mAP |
| 4 | Semantic Segmentation | Encoder-decoder, dilated convolutions, ASPP, loss functions, mIoU |
| 5 | Instance Segmentation | Mask R-CNN, panoptic segmentation, SAM, query-based methods |
| 6 | Object Tracking | Kalman filter, Hungarian algorithm deep dive, Re-ID, SORT β ByteTrack |
| 7 | Pose Estimation | Keypoints, heatmaps, PAF deep dive, top-down vs bottom-up, 3D pose |
| 8 | Face Recognition | Detection, alignment, embedding, ArcFace/CosFace/SphereFace deep dive |
| 9 | Depth Estimation | Stereo, monocular depth, LiDAR fusion, depth completion |
| 10 | Optical Flow | Lucas-Kanade, FlowNet, RAFT, video stabilization |
| 11 | Video Understanding | Action recognition, temporal models (3D conv, transformers), action localization |
| 12 | 3D Vision | Point clouds, NeRF, 3D Gaussian splatting, 3D object detection, SLAM |
| 13 | Anomaly Detection | One-class learning, PatchCore, flow-based, reconstruction-based |
| 14 | OCR & Document Understanding | Text detection, CRNN+CTC, end-to-end OCR, LayoutLM |
| 15 | Medical Imaging | Segmentation, classification, domain challenges, foundation models (MedSAM) |
| 16 | Multimodal Vision Tasks | VQA, image captioning, visual grounding, text-to-image generation |
| Architectures | Β | Β |
| 17 | CNN Fundamentals | Convolution math, receptive fields, normalization, depthwise & dilated convs |
| 18 | Classification Architectures | ResNet, MobileNet, EfficientNet, ViT, Swin, ConvNeXt β with param/FLOP tables |
| 19 | Detection Architectures | YOLO (v1βv11), Faster R-CNN, DETR, RT-DETR, FCOS, CenterNet |
| 20 | Segmentation Architectures | FCN, U-Net, DeepLab v3+, Mask R-CNN, Mask2Former, SAM |
| 21 | Pose Architectures | Stacked Hourglass, HRNet, ViTPose, OpenPose, DEKR, WholeBody |
| 22 | Face Architectures | MTCNN, RetinaFace, MobileFaceNet, ArcFace, AdaFace |
| 23 | Tracking Architectures | SORT, DeepSORT, FairMOT, ByteTrack, StrongSORT, TrackFormer |
| 24 | Vision Transformers | ViT, DeiT, Swin, MAE, DINO, DINOv2 β attention in 2D |
| 25 | Multimodal Architectures | CLIP, BLIP-2, LLaVA, Grounding DINO, SAM 2, Flamingo |
| Optimization | Β | Β |
| 26 | Optimization Overview | The accuracy-speed-size triangle, profiling, optimization taxonomy |
| 27 | Quantization | PTQ, QAT, INT8/FP8, mixed precision, per-channel vs per-tensor |
| 28 | Pruning & Sparsification | Structured/unstructured pruning, lottery ticket, movement pruning, ASP |
| 29 | Knowledge Distillation | Feature mimicking, attention transfer, task-specific KD, self-distillation |
| 30 | Neural Architecture Search | RL/evolutionary/gradient-based NAS, DARTS, Once-for-All, EfficientNet |
| 31 | Efficient Inference | Graph fusion, ONNX, TensorRT, OpenVINO, TFLite, CoreML, NCNN |
| Data & Training | Β | Β |
| 32 | Datasets | ImageNet, COCO, ADE20K, LFW, Kinetics, KITTI, MVTec β every major benchmark |
| 33 | Data Augmentation | MixUp, CutMix, Mosaic, AutoAugment, Copy-Paste, TTA |
| 34 | Training Recipes | Loss functions, AdamW/LARS/SAM optimizers, LR schedules, EMA, bag-of-tricks |
| 35 | Self-Supervised Learning | SimCLR, MoCo, BYOL, MAE, DINO β learning without labels |
| Deployment & Ecosystem | Β | Β |
| 36 | Deployment | ONNX export, serving (Triton/TorchServe), edge (Jetson/mobile), batching |
| 37 | Ecosystem & Libraries | timm, MMDetection, Detectron2, Ultralytics, OpenCV, Hugging Face, Roboflow |
| 38 | Evaluation & Benchmarks | mAP computation, OKS, HOTA, FID, Rank-1, FLOP counting |
| Frontier | Β | Β |
| 39 | VisionβLanguage Models | CLIP ecosystem, GPT-4V, Gemini Vision, VQA, grounding, generation |
| 40 | Foundation Models | SAM, DINOv2, Florence-2, InternVL β universal visual features |
| 41 | Generative Vision | GANs, VAEs, DDPM, Stable Diffusion, ControlNet, video generation |
| 42 | The Frontier | World models, 3DGS, embodied AI, neuromorphic vision, open problems |
| Appendices | Β | Β |
| A | History of Computer Vision | Full timeline from Hubel & Wiesel to SAM 2 β every milestone and paradigm shift |
| B | Math Foundations | Linear algebra, convolutions as matrix ops, Fourier, probability for CV practitioners |
| C | Glossary | Every key term, acronym, and metric defined in one place |
πΊοΈ Learning Paths
Path A: βI want to build a CV applicationβ (8 chapters)
- 00 β Grand Overview β the big picture
- 01 β Image Fundamentals β images as tensors
- 02 β Image Classification β the simplest task
- 03 β Object Detection β the most deployed task
- 18 β Classification Architectures β choosing your backbone
- 19 β Detection Architectures β YOLO and friends
- 33 β Data Augmentation β training data strategy
- 36 β Deployment β getting to production
Path B: βI want to go deep on a specific taskβ (task chapter β architecture chapter β eval chapter)
- Detection: 03 β 19 β 38
- Segmentation: 04 + 05 β 20 β 38
- Face Recognition: 08 β 22 β 38
- Tracking: 06 β 23 β 38
- Pose Estimation: 07 β 21 β 38
Path C: βI want to optimize for edge/mobileβ (7 chapters)
- 17 β CNN Fundamentals β understand what youβre optimizing
- 18 β Classification Architectures β MobileNet, EfficientNet
- 26 β Optimization Overview β the landscape
- 27 β Quantization β INT8 and beyond
- 28 β Pruning β structured pruning
- 29 β Knowledge Distillation β compress with teachers
- 31 β Efficient Inference β TensorRT, TFLite, CoreML
Path D: βI want to understand vision transformers and foundation modelsβ (6 chapters)
- 24 β Vision Transformers β ViT to DINOv2
- 25 β Multimodal Architectures β CLIP, BLIP-2, LLaVA
- 35 β Self-Supervised Learning β MAE, DINO
- 39 β VisionβLanguage Models β CLIP ecosystem
- 40 β Foundation Models β SAM, DINOv2
- 41 β Generative Vision β diffusion models
Path E: βI want the research frontierβ (full guide)
Read chapters 0 through 42 in order. Appendices A, B, and C provide depth on history, math, and terminology.
π Prerequisites
- Python β NumPy, basic PyTorch tensor operations
- Linear algebra β matrix multiplication, eigenvectors, SVD
- ML basics β loss functions, gradient descent, backpropagation, CNNs
- Probability β softmax, cross-entropy, Gaussian distributions
- Command line β terminal, pip/conda, CUDA setup
π§ How This Guide Is Organized
This guide spans six dimensions of computer vision:
π Changelog
| Date | Changes |
|---|---|
| May 2026 | Initial structure β 43 chapters + 3 appendices |
Last updated: May 2026