Where the graph gets cut

torch_xla.sync() is a small Python function that hands off to a C++ step marker, and the step marker does three things. It syncs the live tensors graph. It bumps a counter and resets the tracing scopes. Then, only under a debug flag, it prints a report about what just happened.

The first of those is the one with teeth. SyncLiveTensorsGraph fetches every live tensor on the device from the arena that tracks them, and syncs all of them with sync_ltc_data true. Not the tensors you mentioned; every tensor that still exists. That is the barrier semantics, and it is why an accidentally retained tensor from a previous step changes the graph rather than being ignored.

Two details in the Python are worth having: metrics are saved only on the master ordinal, so you are not reading N copies of the same numbers, and the step closures registered with add_step_closure run after the marker, which is what lets several closures share one execution instead of forcing one each.

def sync(wait: bool = False, reset_scope: bool = True):
  """Launches all pending graph operations.

  Args:
    wait (bool): whether to block the current process until the execution finished.
    reset_scope (bool): whether to reset the torch::lazy::ScopeContext of the IR Nodes.
  """
  if xu.getenv_as('XLA_EMIT_STEPLOG', bool, False):
    print('torch_xla.torch_xla::sync\n', end='', file=sys.stderr, flush=True)
  torch_xla._XLAC._xla_step_marker(
      torch_xla._XLAC._xla_get_default_device(), [],
      wait=xu.getenv_as('XLA_SYNC_WAIT', bool, wait),
      reset_scope=reset_scope)
  # Only emit metrics from the first local device index, to avoid emitting the
  # same values from different threads.
  if xm.is_master_ordinal():
    xm.ms.save_metrics()
  devctx = xm._run_step_closures()
  torch_xla._XLAC._set_all_reduce_token(devctx.device, None)

Five entry points force materialization and all five funnel into SyncTensorsGraphInternal. What separates them is two booleans on a config struct, and those booleans are not bookkeeping: one of them enters the hash, so the same graph reached by two different roads compiles twice.

sync_ltc_data says whether the synced tensors should be turned into device data afterward. force_ltc_data says whether this sync may alias input buffers into output buffers. A step marker sets both. A print() sets neither, because reading a value must not consume the buffer you are about to read.

Since force_ltc_data is merged into the hash before any tensor is examined, printing a graph and syncing the same graph land on two different cache entries and pay for two compiles. The comment in the source says so in one line, and the reason is aliasing: a sync at a barrier may donate an input buffer to an output, and a print may not, so the two cannot execute the same module.

The knock-on effect shows up in a shape people write without thinking. Debug a loop by printing a loss every iteration and you have added a second compiled graph to every step, permanently, not just while you are watching. Remove the print and the second entry stops being touched, though it stays in the LRU taking a slot.

The buffer donation half of that story has its own home. Which parameters may be donated, and why the answer is only safe at a barrier, is the fourth lesson in this arc; the twenty-four-line comment inside GetBufferDonors is where the source argues it out with two examples.

Beyond the five explicit roads, a handful of ordinary-looking lines cut a graph. Python control flow on a tensor value is the obvious one: if x[0][0] == 3 needs a Python bool, so the graph runs to produce it, and the design essay in the repo is blunt that this cannot be fixed without lowering control flow into the graph.

Calling tensor.size(d) on a dynamic-shape tensor is the same story with a smaller footprint. The docs put it plainly: the op forces XLAThe compiler: brilliant at fusing along dataflow edges, structurally unable to change your algorithm. That gap is why kernels exist.taught in /l/xla → to cut the graph and evaluate, because the alternative is returning the padded shape, which is wrong.

Then there are two that leave no trace in your source at all. Any op without a lowering hits the boxed CPU fallback, which is a device round-trip and therefore a materialization, counted per op in a fallback counter you can read. And mark_sharding on a tensor whose IR is a DeviceData node downloads the buffer to host and re-uploads it as shards, so an annotation that reads like metadata is a synchronous round-trip. On an intermediate tensor the same call is a pure IR annotation and costs nothing.

One category of tensor is deliberately excluded from the step barrier, and the eighteen-line comment above ApplyEagerSync explains why. View tensors are not synced, because updating the base tensor replays every view on it anyway, so syncing views would compile work that is about to be redone.

The comment states the consequence and accepts it: users who want to print a view still can, and will incur a small graph compile. An extra compile you did not ask for, chosen on purpose to avoid a larger set of compiles you also did not ask for. Worth knowing before you file it as a bug.

Underneath, torch_xla is running two view systems at once. The legacy View and Alias machinery with its generation counter coexists with PyTorch's own functionalization dispatch key, and roughly a dozen comments in tensor.h mark the migration as unfinished. That is the kind of detail that makes a stack trace legible when nothing else does.

torch_xla ships its own explainer for all of this. Set PT_XLA_DEBUG=1 and every graph execution walks the Python frames and classifies itself into one of eight causes, from the parallel loader's step end to a profiler region exiting to the dynamo bridge processing input graphs. The final else is the one you will meet most often, and its wording is a good summary of the whole lesson.

The report carries the graph name, the graph hash, the input and output counts, and up to eight Python frames, capped by PT_XLA_DEBUG_MAX_FRAME. At debug level 1 only compilation analyses print; the environment variable set to 1 actually implies level 100, so the loud version is the default one.

Reading this report is the difference between guessing and knowing which line cut your graph. It is also the mechanism, not just the diagnostic, behind full_graph=True, which the next lesson gets to.

  } else if (frames[0].function == "extract_graph_helper" &&
             endsWith(frames[0].file, "dynamo_bridge.py")) {
    ss << debug_output_prefix << "  dynamo is compiling a FX graph to HLO\n";
  } else {
    // TODO(JackCaoG): be more specific about  exeuction caused by printing
    // tensor or fallback or some weird indexing.
    ss << debug_output_prefix
       << "  most likely user code trying to access tensor value before "
          "torch_xla.sync\n";
  }