torch Tensor.contiguous()


a = torch.Tensor([[1,2,3],[5,6,7]])
a.is_contiguous() # True
a.transpose(0,1).is_contiguous() # False
a.transpose(0,1).contiguous().is_contiguous() # True

Debugging history. Encoder weights not updating → Only occurs with Adam (SGD works fine) → Discovered MPS backend bug with non-contiguous tensor handling.

the bug that taught me more about PyTorch than years of using it

a loss plateau that looked like my mistake turned out to be a PyTorch bug. tracking it down meant peeling back every layer of abstraction, from optimizer internals to GPU kernels.

https://elanapearl.github.io/blog/2025/the-bug-that-taught-me-pytorch/

torch Tensor.contiguous()

Recommendations