Seonglae ChoLLM File Types
Local LLMs
No, local models will not win
Every time a new open-weight AI model is released, people say that local models are the future. Why spend billions of dollars building out datacenters when everyone will just be able to run AI models on their laptops or phones? I think this idea is doomed. No matter how strong open-weight models get, most inference will always happen in AI datacenters.
https://www.seangoedecke.com/local-models-will-not-win/
Memory Usage
Model Memory Utility - a Hugging Face Space by hf-accelerate
Discover amazing ML apps made by the community
https://huggingface.co/spaces/hf-accelerate/model-memory-usage
