Superword BPE SuperBPE: Space Travel for Language ModelsThe assumption across nearly all language model (LM) tokenization schemes is that tokens should be subwords, i.e., contained within word boundaries. While providing a seemingly reasonable...https://arxiv.org/abs/2503.13423
Seonglae ChoSeonglae ChoSeonglae ChoSeonglae ChoSeonglae ChoSeonglae Cho
