a mastery course · the pytorch path · the eager world, landing on tpu
Learn to think in PyTorch.
PyTorch is the other philosophy: state is the model, the tape is dynamic, and compilation is opt-in. Part i builds that model until the museum's errors are predictable. Part ii is the practice, and it ends where this site lives: torch running on TPU, over bridges that lower into the same compiler the XLA path opens.
your progress 0/12 chapters
How PyTorch thinks: storage and strides, the dynamic tape, stateful modules, the loop you own, and the compilers that recover a graph from all of it.
- ch 01 Tensors A tensor is a view onto a block of storage, not the numbers themselves, and the strides are the map that reads it.
- ch 02 Autograd The autograd tape isn't traced once ahead of time; it's built fresh, node by node, every time your Python actually runs.
- ch 03 Modules An nn.Module isn't a function you call with parameters; it's a Python object that owns its parameters and remembers them between calls.
- ch 04 Own the state, own the loop The loop that trains a PyTorch model is five lines you write by hand, and every one of them exists because some piece of state needed managing on purpose.
- ch 05 The input pipeline A DataLoader is not part of the model. It's the concurrent program standing between your disk and your device, and it runs by its own rules.
- ch 06 Guarded capture torch.compile doesn't trace your program once and lock it in. It captures your tensor math behind a set of guards and lets everything else keep running as plain Python.
- ch 07 Graphs Whatever captures your model, dynamo's guarded graph or torch.export's whole program, what's underneath is the same ATen vocabulary eager PyTorch already speaks.
Ranks and meshes, honest measurement, then the destination: the two bridges to TPU and a capstone run across one of them.
- ch 08 Distributed A PyTorch distributed job has no separate orchestrator: every process runs the same script, and a collective is the one moment those processes actually talk to each other.
- ch 09 Performance A benchmark number is only as honest as the clock behind it, and PyTorch gives you three different ways for that clock to lie.
- ch 10 Bridges PyTorch never runs on a TPU directly; it crosses one of two bridges first, and by the time either one lands, the framework you wrote in is no longer the point.
- ch 11 TPU practice Once a torch model crosses either bridge, the TPU treats it exactly the way it treats jax: same mesh, same profiler, same kernels underneath.
- ch 12 The training run Nothing in this chapter is new. The training run is what the other eleven chapters were for.
Every page of the official Learn-the-Basics series and the intermediate tutorials this path covers is assigned reading somewhere, and the chapters keep going: the stride map, the guard set, the two-bridge ledger, the benchmarking method.
Every snippet in part i ran before it was printed (torch 2.2.2 on CPU, this machine, 2026-07-27), and printed values are real outputs. The TPU chapters say plainly that their hardware claims await the Colab lab pass; nothing pretends otherwise.
The bridges land on StableHLO, where the XLA path picks up; the JAX path is the philosophy this one contrasts against, named in every chapter that differs; the pytorch gym floor drills the stride map with real torch outputs.