The Annotated S4
The Structured State Space
for Sequence Modeling (S4) architecture is a new approach to very
long-range sequence modeling tasks for vision, language, and audio,
showing a capacity to capture dependencies over tens of thousands of
steps. Especially impressive are the model’s results on the challenging
Long Range
Arena benchmark, showing an ability to reason over sequences of up
to 16,000+ elements with high accuracy.
https://srush.github.io/annotated-s4/
