🔬 Chips — The Complete Guide
From Sand to Supercomputers: a first-principles guide to chip design and architecture, written for the AI/ML practitioner who optimizes models. By the end you will understand the hardware reasons behind every quantization, sparsity, batching, and parallelism decision you make.
Who This Is For
You train, fine-tune, quantize, prune, or serve neural networks. You are fluent in Python and PyTorch, comfortable with tensors, dtypes, and shapes — but the silicon underneath is a black box. Why is bf16 better than fp16 for training? Why does INT8 give a ~2× speedup but INT4 sometimes doesn’t? Why does a 70B model refuse to fit on one 80 GB GPU? Why is memory bandwidth — not FLOPs — usually what you’re actually waiting on? Why did 2:4 sparsity become a thing, and why not 3:7?
This guide answers all of that from first principles — starting at the transistor and building all the way up to multi-GPU training clusters and the post-Moore future. It is self-contained: every concept is explained here, from the ground up.
What This Guide Covers
Foundations
Transistors, logic, number systems, processors, ISAs, the vocabulary of the stack
Memory & Movement
SRAM/DRAM/HBM, caches, the memory wall, roofline, buses & interconnects
The Chips
CPU, GPU, TPU, NPU, DSP, ISP, FPGA, ASIC, SoC — what each is and why
Datatypes
FP32→FP4, bit layouts, microscaling, and how formats reshape silicon
Optimization
Quantization & sparsity from the hardware's point of view — and their limits
Hardware → Usage
Why GPUs won AI, multi-GPU training, KV-cache, and the industry's future
📋 Table of Contents
| # | Chapter | What You’ll Learn |
|---|---|---|
| Part I — Foundations: What Is a Chip? | ||
| 1 | What Is a Chip? | Sand → silicon → transistors; the MOSFET, doping, Moore’s Law, Dennard scaling |
| 2 | Transistors to Logic | Boolean algebra, gates, combinational vs sequential logic, flip-flops, the ALU |
| 3 | Number Representation | Binary, two’s complement, fixed vs floating point, IEEE-754 — with bit-inspection code |
| 4 | Anatomy of a Processor | von Neumann vs Harvard, fetch-decode-execute, datapath, registers, the clock, pipelining |
| 5 | Instruction Sets | ISA vs microarchitecture, RISC vs CISC, x86/ARM/RISC-V, SIMD & vector extensions |
| 6 | Languages & Nomenclature | Bare metal, firmware, microcode, assembly, Verilog/VHDL/RTL, HLS, CUDA, ROCm, Triton |
| 7 | Types of Chips (Taxonomy) | The full map: CPU/GPU/TPU/NPU/DSP/ISP/FPGA/ASIC/SoC/MCU in one comparison |
| Part II — Parallelism, Memory & Movement | ||
| 8 | Parallel Processing | ILP, superscalar, SIMD/SIMT, multicore, Flynn’s taxonomy, Amdahl vs Gustafson |
| 9 | Memory Types | Registers, SRAM, DRAM, DDR/GDDR/HBM, NAND; the hierarchy, latency & bandwidth numbers |
| 10 | Memory Management | Virtual memory, paging, MMU/TLB, cache coherence (MESI), NUMA, consistency |
| 11 | Memory Wall & Bandwidth | The roofline model, arithmetic intensity — why bandwidth, not FLOPs, bounds ML |
| 12 | Buses & Interconnects | On-chip (NoC, AXI) vs off-chip (PCIe, CXL, NVLink, UALink); topologies & bandwidth |
| 13 | Reliability & Fault Tolerance | ECC, RAS, soft errors, silent data corruption, redundancy — and long training runs |
| Part III — The Chips Themselves | ||
| 14 | CPUs | Out-of-order superscalar cores, branch prediction, caches; when/why CPUs for ML |
| 15 | GPUs | SIMT, SMs, warps, tensor cores, memory hierarchy — why GPUs won ML |
| 16 | TPUs & Systolic Arrays | Systolic arrays, matrix-multiply units, dataflow; Google TPU v1→Trillium |
| 17 | NPUs & Edge AI | Mobile/edge NPUs, Apple Neural Engine, Qualcomm Hexagon, Renesas DRP-AI |
| 18 | DSPs | MAC units, VLIW, signal-processing pipelines; where DSPs live |
| 19 | ISPs | Image signal processors, the camera pipeline, their place in SoCs |
| 20 | FPGAs | LUTs, reconfigurable fabric, when/why FPGAs, AI on FPGA |
| 21 | ASICs | The FPGA→ASIC spectrum, NRE/full-custom, the AI-ASIC boom |
| 22 | SoCs | Integration, chiplets, heterogeneous compute, Apple M-series, mobile SoCs |
| Part IV — Datatypes (the keystone) | ||
| 23 | Number Formats Deep Dive | FP32→FP4, INT8→INT1, bit layouts, microscaling (MXFP, NVFP4) — with code |
| 24 | Datatypes & Silicon | How a format reshapes the multiplier, tensor core, and per-generation HW support |
| Part V — Optimization (HW ↔ model co-design) | ||
| 25 | Quantization (Hardware View) | How low precision saves area/power/bandwidth — to what extent, and where it breaks |
| 26 | Sparsity (Hardware View) | Structured 2:4 sparsity, sparse tensor cores — to what extent it actually helps |
| Part VI — From Hardware to Usage | ||
| 27 | Why GPUs Won AI | How GPU parallelism maps to neural-net math; matmul as the workhorse |
| 28 | Multi-GPU & Distributed Training | Data/tensor/pipeline/expert parallelism, FSDP, ZeRO, 3D — why 70B won’t fit on one GPU |
| 29 | Memory Hierarchy in Action | How HW memory limits drive KV-cache, batching, activation memory, offloading |
| 30 | Datatypes Drive Optimization | A practitioner’s decision framework: when INT8, when FP8, when 2:4 sparsity |
| Part VII — How Chips Are Designed, Verified & Tested | ||
| 31 | Chip Design Flow | Spec → RTL → synthesis → place & route → timing → DRC/LVS → tapeout; the EDA stack |
| 32 | Verification & Validation | Simulation, UVM, assertions, formal, coverage, DFT, emulation, post-silicon — with examples |
| Part VIII — The Industry: Players, Evolution & Future | ||
| 33 | Design-Paradigm Wars | NVIDIA vs AMD vs Intel vs ARM — design philosophies and how the lead changed hands |
| 34 | Generational Improvements | Process nodes, bandwidth growth, and tabulated family histories year over year |
| 35 | Where the Industry Is Headed | Chiplets, 3D stacking, CXL pooling, co-packaged optics, analog/in-memory, the post-Moore era |
| Appendices | ||
| A | How Chips Are Made | Sand → wafer → fab → EUV lithography → etch/doping → packaging → test; yield & binning |
| B | Foundry Ecosystem | TSMC/Samsung/Intel, the fabless model, EDA tools, IP cores, the supply chain |
| C | Photonics | Silicon photonics, co-packaged optics, optical interconnect & compute |
| D | Power & Thermals | Power delivery, DVFS, cooling, performance-per-watt, the energy cost of AI |
| E | Packaging & Integration | 2.5D/3D, CoWoS, interposers, HBM stacking, chiplets/UCIe |
| F | Glossary | Every acronym and term in one place |
| G | Resources | Books, courses, channels, blogs, papers, podcasts for going deeper |
| H | The Life of a Running Computer | What happens at power-on, how interrupts & the scheduler work, what “idle” (HLT/C-states) really is, and who keeps the screen lit |
🗺️ Learning Path
Recommended Learning Path
🧱 Ch 1–7: Foundations — transistors to ISAs to the chip taxonomy
🧠 Ch 8–13: Parallelism, memory, and how data moves
⚙️ Ch 14–22: The chips themselves, one architecture at a time
🔢 Ch 23–24: Datatypes — the keystone of optimization
🗜️ Ch 25–26: Quantization & sparsity from the silicon's view
🚀 Ch 27–30: Hardware → usage (why GPUs won, multi-GPU, KV-cache, your choices)
🏭 Ch 31–35 + Appendices: Design, verification, the giants, the future, fabrication
⚡ Quick Start Paths
Path A: “I optimize models and want to know why my choices work” (6 chapters)
- 11 — Memory Wall & Bandwidth — the roofline; why you’re memory-bound
- 23 — Number Formats Deep Dive — what FP8/INT4 really are
- 25 — Quantization (Hardware View) — how much it actually buys you
- 26 — Sparsity (Hardware View) — when 2:4 helps
- 29 — Memory Hierarchy in Action — KV-cache & batching
- 30 — Datatypes Drive Optimization — the decision framework
Path B: “I want to understand the hardware end-to-end” (full guide)
Read Chapters 1 → 35 in order, then the appendices. Each builds on the previous.
Path C: “I’m scaling training to many GPUs” (5 chapters)
- 09 — Memory Types — HBM and the memory hierarchy
- 12 — Buses & Interconnects — NVLink, PCIe, the fabric
- 15 — GPUs — what’s inside the accelerator
- 27 — Why GPUs Won AI — the hardware/math fit
- 28 — Multi-GPU & Distributed Training — DP/TP/PP/FSDP/ZeRO
Path D: “I’m curious how chips are actually built” (4 reads)
- 31 — Chip Design Flow — RTL to GDSII
- 32 — Verification & Validation — how correctness is proven
- Appendix A — How Chips Are Made — the physical fab
- Appendix B — Foundry Ecosystem — the industry behind it
📚 Prerequisites
You’ll get the most out of this guide if you’re comfortable with:
- Python & PyTorch — tensors, dtypes, shapes, basic
nn.Moduleusage - Basic ML — what training, inference, and a matrix multiply are
- A little binary — bits and bytes (Chapter 3 reteaches everything else)
- Curiosity about hardware — no electrical-engineering background required
Everything else — semiconductor physics, logic design, computer architecture, HDLs — is taught from scratch.
🏗️ How This Guide Is Built
Every chapter follows a What → Why → Which → When → Where rhythm, and shares a consistent toolkit:
Chapter Toolkit
First Principles
Concepts built up from the transistor, never assumed
Tables & Diagrams
Comparison tables and CSS diagrams for every key idea
Math Where It Helps
Roofline, bandwidth, area/power — derived, not hand-waved
Python/PyTorch Code
Runnable snippets; low-level languages where they're the topic
Dtypes & Shapes
Bit layouts, byte sizes, tensor shapes, bandwidth numbers, always annotated
ML Linkage
Every hardware idea tied back to a real model-optimization decision
📝 Changelog
| Date | Changes |
|---|---|
| May 2026 | Initial release — 35 chapters + 7 appendices |
Last updated: May 2026 · Built with Jekyll, deployed on GitHub Pages.