Scalable Parallelization for ML Computation GraphsTo enable FSDP via SPMD PyTorch on XLA Devices — PyTorch/XLA master documentationPyTorch runs on XLA devices, like TPUs, with the torch_xla package. This document describes how to run your models on these devices.https://pytorch.org/xla/release/2.1/index.html#pytorch-xla-spmd-user-guidearxiv.orghttps://arxiv.org/pdf/2105.04663.pdf
