Seonglae ChoMore than just inference, it's a model loading and training tool utility where using Flash Attention is the real accelerate
several backends
pip install accelerate accelerate config # or python -c "from accelerate.utils import write_basic_config; write_basic_config(mixed_precision='fp16')"
Huggingface accelerate usages
Installation and Configuration
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/docs/accelerate/basic_tutorials/install
