Seonglae ChoState Space Model
RNN can also be viewed as a type of SSM.
- The concept of state space models was developed in the 1960s in the aerospace and control engineering fields to analyze and design dynamic systems. It mathematically represents the relationship between the current state and the next state of a system, serving as an important tool for predicting and controlling the dynamic behavior of systems.
- In the 1970s, statisticians began adopting state space models to analyze time series data. These models were useful for exploring the structure of data that changes over time and modeling uncertainty.
SSMs
SSM Notion
It is more wide concept for differentiable programming
State-space representation
In control engineering and system identification, a state-space representation is a mathematical model of a physical system specified as a set of input, output, and variables related by first-order differential equations or difference equations. Such variables, called state variables, evolve over time in a way that depends on the values they have at any given instant and on the externally imposed values of input variables. Output variables’ values depend on the state variable values and may also depend on the input variable values.
https://en.wikipedia.org/wiki/State-space_representation
A Visual Guide to Mamba and State Space Models
An Alternative to Transformers for Language Modeling
https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-and-state#footnote-anchor-3-141228095
