The four numbers
A TPU v5e: 1.97e14 bf16 FLOPs per second in the MXU, 8.2e11 bytes per second from HBM, roughly 128 MiB of VMEM scratchpad, 16 GB of HBM. Every kernel decision in this track is arithmetic over these four numbers. The ratio of the first two is the ridge (about 240 FLOPs per byte): ops below it are memory-bound, ops above it compute-bound, and no cleverness moves an op across the ridge except changing its algorithm.
The MXU is a 128x128 systolic array (256x256 on v6e): matmuls only. The VPU handles everything elementwise in (8, 128) lanes, which is where the tiling lattice that haunts every layer above actually comes from. VMEM is software-managed: nothing is cached for you; what is resident is what your schedule staged. Stage 0's EX·06 draws the hierarchy to these real widths, and EX·01 lets you drag op shapes across the roofline.
The habit the whole track builds on this layer: predict from the constants before measuring, then explain every miss. The constants are public, the arithmetic is short, and an engineer who cannot predict a latency floor from them is guessing everywhere above.
Lessons
- 01The TPU chipA TPU is one bet cast in silicon: almost all of the work is matrix multiplication, so build one enormous unit for exactly that and spend the rest of the chip feeding it. ·
- 02Six generations of the same ideaEvery TPU generation is the same machine with different constants. The constants are the personality, and you can read a generation like a datasheet once you know which ratios matter. ·
- 03The TensorCore complexThe v4 chip is the last TPU with a paper behind it, and that paper prints enough parts to rebuild the headline number from scratch. ·
- 04SparseCoreEvery TPU since v2 carries a third kind of core that no kernel in this course touches. It exists because one workload class was worth 5% of the die rather than a rewrite of the array. ·
- 05Two ways to hide latencyA GPU hides memory latency with thousands of threads. A TPU has one big core and no threads to switch, so every wait has to be planned away. ·
- 06XProf, and the three accountsThe profiler is how prediction meets the machine, and a program has three cost tellings that this site once caught disagreeing in public. ·
- 07The timeline, op by opOne plane of the trace matters for kernel work: the device plane, where every event carries a name you already know how to read. ·
- 08Reading it like an operatorSix habits turn a wall of events into a diagnosis, and the first is always the same: name the envelope before judging anything inside it. ·
- 09The bundle, and the floor below itOne TPU instruction is 322 bits wide and speaks to several units at once. Filling those slots is the compiler’s whole job, and the layer where it happens is the one you cannot read. ·