Model Qunatization 되는 애들도 많이 없다 오류 많이 난다 특히
이오류 많이난다
openlm-research/open_llama_3b or psmathur/orca_mini_3b 이건 quantization을 되는데 사이즈 작아서 안된다
seonglae/opt-125m-4bit-gptq · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/seonglae/opt-125m-4bit-gptq
이건 quantization 되는데 다시 불러오면 아래같은 오류나옴 뭐지..
seonglae/tulu-7b-4bit-gptq · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/seonglae/tulu-7b-4bit-gptq
Seonglae Cho