Hidden State

Creator

Creator

Seonglae Cho

Created

Created

2023 Mar 7 13:50

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Jun 8 15:45

Refs

Refs

Hidden vector, Thought vector

latent state, internal state

사람이 읽지 못하기 때문에 hidden이라고 함

The hidden state is computed based on the current input and the previous hidden state

이 프로세스는 순차적이었고 병렬화를 방해

Some Intuition on Attention and the Transformer

What's the big deal, intuition on query-key-value vectors, multiple heads, multiple layers, and more.

Some Intuition on Attention and the Transformer

https://eugeneyan.com/writing/attention/

Some Intuition on Attention and the Transformer

Recommendations

///////