Accelerate Multi-GPU on several Nodes How to
Hi, I wonder how to setup Accelerate or possibly train a model if I have 2 physical machines sitting in the same network. Each machine has 4 GPUs. Can I use Accelerate + DeepSpeed to train a model with this configuration ? Can’t seem to be able to find any writeups or example how to perform the “accelerate config”. Thanks.
https://discuss.huggingface.co/t/accelerate-multi-gpu-on-several-nodes-how-to/10736