Seonglae Chohybrid architecture with Mamba and Attention heads running in parallel
outperforms popular small language with 6-12x less training
Hymba - a nvidia Collection
A series of Hybrid Small Language Models.
https://huggingface.co/collections/nvidia/hymba-673c35516c12c4b98b5e845f
