Seonglae Cho- GPTQ for int4
- bitsandbytes for int8, fp4, nf4
not available for
Download mode and so annoying~
mac
Why We're Building Lit-LLaMA - Lightning AI
The Age of Personal AI Large language models (LLMs) have rapidly gained popularity across the globe. These models are trained to predict the next word in extensive datasets of raw text, exhibiting emergent abilities to complete tasks they were not specifically trained for (e.g., summarizing a sentence in Shakespearean style or playing the next move... Read more »
https://lightning.ai/pages/community/why-were-building-lit-llama/
