Texonom
Texonom
/
Science
Science
/Mathematics/Math Field/Statistics/Statistical Model/Model Generalization/Model Training/Model Training Tool/Huggingface accelerate/
Huggingface distributed inference
Search

Huggingface distributed inference

Creator
Creator
Seonglae Cho
Created
Created
2023 Nov 20 13:52
Editor
Editor
Seonglae Cho
Edited
Edited
2024 Mar 17 6:48
Refs
Refs
 
  1. device map
if you just want distributed memory
  1. PartialState
 
 
 
 
Distributed inference with multiple GPUs
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Distributed inference with multiple GPUs
https://huggingface.co/docs/diffusers/training/distributed_inference
Distributed inference with multiple GPUs
Handling big models for inference
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Handling big models for inference
https://huggingface.co/docs/accelerate/usage_guides/big_modeling
Handling big models for inference
 
 

Recommendations

Texonom
Texonom
/
Science
Science
/Mathematics/Math Field/Statistics/Statistical Model/Model Generalization/Model Training/Model Training Tool/Huggingface accelerate/
Huggingface distributed inference
Copyright Seonglae Cho