Graphs

Dynamo captures your tensor math into a graph, chapter 6's whole subject, but the graph itself is not exotic. It's FX, and every node in it calls an ATen op: the same operator library eager PyTorch calls under the hood, whether or not dynamo ever ran. Compile a function and print its captured ops, and you're reading the same vocabulary a plain, uncompiled forward pass already spoke.

That graph doesn't stop at the forward pass either. aot_autograd traces the backward through the same machinery, so what dynamo hands the compiler already covers a full training step, forward and backward both, one FX graph each, not something bolted on after the fact. This is why torch.compile speeds up training loops, not only inference calls.

One more pass runs before the graph settles: decompositions rewrite the larger ATen surface down toward a smaller core set of ops, fewer signatures for a compiler backend to implement, more of them expressible as combinations of the ones that remain. A backend author who supports the core set gets most of the rest for free.

Below dynamo, everything is ATen.

Dynamo's graphs are stitched: wherever a guard might miss or Python control flow resists capture, a graph break splits the function into pieces and Python fills the gap between them, chapter 6's whole story. torch.export refuses that compromise. Call torch.export.export on a module and you get back an ExportedProgram: one standalone graph for the whole function, with no Python left in it at all, and no seam where a break could have happened.

That completeness is what makes an ExportedProgram portable. It's the artifact you hand to a serving stack running with no Python interpreter alongside it, and it's the same artifact chapter 10 picks up and carries across the bridge to StableHLOThe portable, versioned tensor IR that JAX and PyTorch both lower into; chapter 03 reads it line by line.taught in /l/stablehlo →, the wire format a compiler on the other side of that bridge actually reads. Where dynamo optimizes the training loop you're already running, export produces something you can hand off entirely.

import torch

class Sin(torch.nn.Module):
    def forward(self, x):
        return torch.sin(x).sum()

ep = torch.export.export(Sin(), (torch.randn(4),))
print(ep.graph)

The call above prints exactly the text below, and reading it takes a habit the jax path already built for jaxprsThe traced program: one equation per primitive in single-assignment form, every shape and dtype stated.taught in /l/jaxpr →: find the placeholders, the inputs coming in, then follow one operation per line down to the outputs returned at the end. The IR is different, ATen calls here, JAX primitives there, but the fluency is the same skill wearing a different vocabulary.

Read it line by line and there's nothing left to guess at. %l_x_ is the placeholder, the function's one input. %sin calls torch.ops.aten.sin.default on it, and %sum_1 calls torch.ops.aten.sum.default on whatever %sin produced. The graph returns sum_1, and that's the whole function: two ATen calls, one input, one output, exactly what Sin.forward wrote in Python, with nothing hidden and nothing implicit left standing between them.

graph():
    %l_x_ : [num_users=1] = placeholder[target=l_x_]
    %sin : [num_users=1] = call_function[target=torch.ops.aten.sin.default](args = (%l_x_,), kwargs = {})
    %sum_1 : [num_users=1] = call_function[target=torch.ops.aten.sum.default](args = (%sin,), kwargs = {})
    return (sum_1,)