1B sequence lengthDilated Attention Microsoft’s LongNet Scales Transformer to One Billion TokensScaling sequence length is of paramount importance for large language models, as it brings about singnificant benefits. These advantages…https://medium.com/syncedreview/microsofts-longnet-scales-transformer-to-one-billion-tokens-af02ff657d87arxiv.orghttps://arxiv.org/pdf/2307.02486.pdf
