1.58bit (-1, 0, 1) dramatic memory efficiencyThe Era of 1-bit LLMs: All Large Language Models are in 1.58 BitsHTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on.https://arxiv.org/html/2402.17764v1Model Olmo NousResearch/OLMo-Bitnet-1B · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.https://huggingface.co/NousResearch/OLMo-Bitnet-1BOLMo-BitnetPublish your model insights with interactive plots for performance metrics, predictions, and hyperparameters. Made by Jeffrey Quesnelle using W&Bhttps://wandb.ai/emozilla/olmo/reports/OLMo-Bitnet--Vmlldzo3MzQ4NjQw?accessToken=t17hxb3r7iedofrmfpog1d3dpbmspp1gz6702uz8gyvokmeovznz5g0ndkhbb6r6Finetune (74mb file, 198 tokens per second on just 1 cpu core)nisten on Twitter / Xhacked bitnet for finetuning, ended up with a 74mb file. It talks fine at 198 tokens per second on just 1 cpu core. Basically witchcraft.opensourcing later via @skunkworks_ai base here: https://t.co/n4iddFDMSl pic.twitter.com/w0u7LJ5S3d— nisten (@nisten) July 31, 2024 https://x.com/nisten/status/1818529201231688139?t=a2_oszg66OrDGlwweQS1iQ&s=19Fine-tuning LLMs to 1.58bit: extreme quantization made easyWe’re on a journey to advance and democratize artificial intelligence through open source and open science.https://huggingface.co/blog/1_58_llm_extreme_quantizationDeepseek R1Run DeepSeek-R1 Dynamic 1.58-bitDeepSeek R-1 is the most powerful open-source reasoning model that performs on par with OpenAI's o1 model. Run the 1.58-bit Dynamic GGUF version by Unsloth.https://unsloth.ai/blog/deepseekr1-dynamic
