Seonglae ChoBlock sparse attention
Not a Full self-attention
RoBERTa
google/bigbird-roberta-large · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/google/bigbird-roberta-large
google/bigbird-base-trivia-itc · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/google/bigbird-base-trivia-itc
vasudevgupta/bigbird-roberta-natural-questions · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/vasudevgupta/bigbird-roberta-natural-questions
google/bigbird-pegasus-large-arxiv · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/google/bigbird-pegasus-large-arxiv
google/bigbird-pegasus-large-bigpatent · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/google/bigbird-pegasus-large-bigpatent
google/bigbird-pegasus-large-pubmed · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/google/bigbird-pegasus-large-pubmed
Documents
Understanding BigBird's Block Sparse Attention
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
https://huggingface.co/blog/big-bird
BigBird, Block Sparse Attention
https://velog.io/@nawnoes/sparse-attention
