Huggingface distributed inference

Creator

Creator

Created

Created

2023 Nov 20 13:52

Editor

Editor

Edited

Edited

2024 Mar 17 6:48

Refs

Refs


accelerate launch

device map

if you just want distributed memory

PartialState

Distributed inference with multiple GPUs

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Distributed inference with multiple GPUs

https://huggingface.co/docs/diffusers/training/distributed_inference

Distributed inference with multiple GPUs

Handling big models for inference

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Handling big models for inference

https://huggingface.co/docs/accelerate/usage_guides/big_modeling

Handling big models for inference

Recommendations

//////////