When a view refuses

The view docstring states the rule before it states the error, and the notation is worth reading slowly. Each new axis must either be a subspace of an old axis, or span a run of old axes d through d+k where every neighbouring pair satisfies one equation: stride[i] equals stride[i+1] times size[i+1].

Read that equation as a question about adjacency. It asks whether stepping one unit along axis i lands exactly where you would arrive by walking all the way through axis i+1. When that holds, the two axes are laid out end to end in storage and merging them into one axis is just relabelling. When it fails, there is a gap or an overlap between them and no single stride can describe the merged axis.

A tensor that is contiguous everywhere satisfies the equation at every pair, which is why any reshape of a contiguous tensor is a view. The equation is weaker than contiguity, though, and the next section is about the gap between them.

The returned tensor shares the same data and must have the same number
of elements, but may have a different size. For a tensor to be viewed, the new
view size must be compatible with its original size and stride, i.e., each new
view dimension must either be a subspace of an original dimension, or only span
across original dimensions :math:`d, d+1, \dots, d+k` that satisfy the following
contiguity-like condition that :math:`\forall i = d, \dots, d+k-1`,

.. math::

  \text{stride}[i] = \text{stride}[i+1] \times \text{size}[i+1]

The implementation states the same rule as a two-step procedure, and this phrasing is easier to run by hand. Cut the old shape into chunks of axes that are contiguous within the chunk. Then check that the new shape can be cut into the same number of chunks with matching element counts, chunk for chunk.

The word doing the work is within. A transposed tensor is not contiguous as a whole, and it still has chunks. Ask for a new shape whose cuts fall on the chunk boundaries and the view succeeds; ask for one whose cuts fall inside a chunk boundary that does not exist, and it cannot.

One line in that loop is worth pointing at, because it is where size-1 axes get their exemption. A chunk boundary is not declared when the neighbouring old axis has size 1, so a size-1 axis never breaks a chunk no matter what stride it carries. The other size-1 exemption, the one inside the contiguity test, is a different line in a different function, and the last lesson of this arc has it.

// On a high level,
// 1. separate `oldshape` into chunks of dimensions, where the dimensions are
//    ``contiguous'' in each chunk, i.e., oldstride[i] = oldshape[i+1] *
//     oldstride[i+1]
// 2. `newshape` must be able to be separated into same number of chunks as
//    `oldshape` was separated into, where each chunk of newshape has matching
//    ``numel'', i.e., number of subspaces, as the corresponding chunk of
//    `oldshape`.

The run below asks three questions of one storage. The first tensor is a slice with a size-1 middle axis, strides (12, 4, 1), not contiguous, and view(2, 4) succeeds on it: the size-1 axis is exempt, the remaining pair satisfies the equation, and the result aliases the original base.

Notice what the successful view produced. Shape (2, 4) with strides (12, 1), which is itself not contiguous. A view is not a promise about the result's layout, only about whether one existed.

The second tensor is a transpose with strides (4, 12, 1). Merging its first two axes needs stride[0] to equal stride[1] times size[1], which is 12 times 2, and 4 is not 24. The view raises with the message the survey chapter quotes, and reshape answers the same request by copying.

import torch

base = torch.arange(24.)
x = base.reshape(2, 3, 4)

b = x[:, 1:2, :]
print(b.stride(), b.is_contiguous())
print(b.view(2, 4).stride(), b.view(2, 4)._base is base)

t = x.transpose(0, 1)
print(t.stride(), t.is_contiguous())
try:
    t.view(6, 4)
except RuntimeError as err:
    print(err)

# (12, 4, 1) False
# (12, 1) True
# (4, 12, 1) False
# view size is not compatible with input tensor's size and stride (at least one
# dimension spans across two contiguous subspaces). Use .reshape(...) instead.

reshape is a short dispatch around the same function. It calls computeStride, and if that returns strides it hands back an alias through a private op that skips view's duplicated work. If computeStride returns nothing, the last line clones and views the clone.

Read that last line closely, because it decides something about the result that nothing in the call site hints at. The clone is taken with at::MemoryFormat::Contiguous, so a reshape that copies always lands on a row-major layout, whatever the input's layout was. A reshape that aliases keeps whatever strides computeStride produced.

That gives you two tensors from one call that differ in ownership, in layout, and in whether a later in-place write is visible to the original. _base is None is the one-line test that tells them apart after the fact, and it costs nothing to add to a debugging session.

The same call returns an alias or an owner, and only the tensor knows which.

  // `computeStride` returns the proper strides to use if this
  // `reshape` can be just a view.
  auto stride = at::detail::computeStride(self.sizes(), self.strides(), shape);

  if (stride.has_value()) {
    // Temporary check to revert to the old behavior/view in cases where the
    // device is not supported (e.g. for XLA the operation is not supported
    // so we use `view` instead).
    //
    // We need to do the checks here instead of in `native_functions.yaml`
    // to preserve backwards compatibility.
    if (!self.is_xla() && !self.is_lazy() && !self.is_ipu()) {
      return self._reshape_alias(shape, stride.value());
    } else {
      return self.view(shape);
    }
  }
  return at::_unsafe_view(self.clone(at::MemoryFormat::Contiguous), shape);

Two view operations that undo each other leave the strides where they started, and the corpus has several rows that do it. A permute followed by a transpose that reverses it, or t() applied twice, both return a contiguous tensor with row-major strides, because the metadata went out and came back.

This matters for a habit rather than for a fact. A long chain of views is not progressively more broken; it is a single stride tuple that got rewritten a few times, and it can land anywhere in the space, contiguous included. The only way to know where it landed is to compute it, which the previous lesson gave you the arithmetic for, or to ask, which the explorer on the chapter page does op by op.