← Back to Table of Contents

Chapter 15 β€” Medical Imaging

β€œIn medical imaging, a single false negative can cost a life β€” the stakes are different from natural images.”

Medical Imaging Modalities

Planned content: X-ray (2D projection, fast, radiation), CT (3D volumetric, cross-sections, higher dose), MRI (3D, soft tissue contrast, no radiation, slow), Ultrasound (real-time, portable, operator-dependent), Pathology slides (WSI: gigapixel images). Each modality has unique characteristics for model design.

πŸ“Š Planned diagram: Medical modality overview β€” X-ray [H, W, 1], CT [D, H, W], MRI [D, H, W], histopathology WSI (multi-resolution pyramid), fundus image [H, W, 3].

Segmentation in Medical Imaging

Planned content: U-Net: the dominant architecture β€” encoder-decoder with skip connections, designed for medical images with limited training data. V-Net: 3D U-Net for volumetric data. nnU-Net: self-configuring U-Net baseline. TransUNet, SwinUNETR: transformer-based. Challenges: thin structures, class imbalance, 3D data, limited annotations.

πŸ“Š Planned diagram: U-Net architecture with tensor shapes β€” input [B, 1, 572, 572] β†’ encoder contracting path β†’ bottleneck [B, 1024, 28, 28] β†’ decoder expanding path β†’ output [B, 2, 388, 388].

Classification in Medical Imaging

Planned content: CheXNet: DenseNet-121 for 14 chest X-ray findings. Diabetic retinopathy grading. Skin lesion classification (ISIC). Pathology patch classification. Multi-label, imbalanced datasets. Grad-CAM visualization for explainability.

Detection and Localization

Planned content: Lung nodule detection (LUNA16). Polyp detection (colonoscopy). Lesion detection. Challenges: variable sizes, very small objects, 3D bounding boxes. DETR and Mask R-CNN adapted for medical images.

Domain Shift and Limited Annotations

Planned content: Domain gap between scanners, institutions, and protocols. Transfer learning from ImageNet vs. medical pre-training. Semi-supervised learning (mean teacher, pseudolabels). Active learning for annotation efficiency. Self-supervised pre-training (DINO on histology).

Foundation Models for Medical Imaging

Planned content: MedSAM: fine-tuned SAM for medical image segmentation. SuPreM: universal pre-training across modalities. BioViL-T: chest X-ray + radiology reports. CONCH: histopathology foundation model. Zero-shot generalization across organs and modalities.

πŸ“Š Planned diagram: Foundation model for medical imaging β€” pre-training on multi-modal data β†’ fine-tuning on specific tasks β†’ comparison with task-specific baselines.

Evaluation in Medical AI

Planned content: Dice Score (segmentation). Hausdorff Distance (boundary quality). AUC-ROC. Sensitivity and specificity. FROC (Free-Response ROC) for detection. Inter-rater agreement (Cohen’s ΞΊ). Regulatory considerations (FDA clearance, CE marking).

Next: Chapter 16 β€” Multimodal Vision Tasks β†’


Last updated: May 2026