Pipeline parallelism

Creator
Creator
Seonglae Cho
Created
Created
2023 Nov 20 15:32
Editor
Edited
Edited
2025 Jan 27 15:32
Refs
Refs

Separate all layers

  • Group sub-parts of the network
  • Challenge to keep all GPU busy
Some utilization between server and on-device could be interesting
Pipeline parallelism Methods
 
 
 
 
 
 
 

Recommendations