GRU

Creator

Creator

Created

Created

2021 Oct 6 10:17

Editor

Editor

Edited

Edited

2024 Oct 21 11:57

Refs

Refs

Long-term dependency

Machine Reading Comprehension

Gating Mechanism

Gated recurrent unit

Selectively forget and retain historical information

Selectively incorporated through

r

and Selectively balanced through

z

with

Hadamard product

r_t = \sigma(W_{x, r} x_t + W_{h, r} h_{t-1} + \alpha_x + \alpha_h) \\ z_t = \sigma(W_{x, z} x_t + W_{h, z} h_{t-1} + \alpha_x + \alpha_h) \\ n_t = g(W_{x, n} x_t + r_t \odot (W_{h, n} h_{t-1} + \alpha_h)) \\ h_t = (1 - z_t) \odot n_t + z_t \odot h_{t-1}

GRUs have significantly more parameters than RNN

LSTM 에서 gate 2개로 줄여서 가중치가 적다. (출력 게이트 없음)

GRU Notion

GRU Update Gate

Gated Recurrent Units (GRU)

Gated Recurrent Units (GRU) GRU는 게이트 메커니즘이 적용된 RNN 프레임워크의 일종으로 LSTM에 영감을 받았고, 더 간략한 구조를 가지고 있습니다. 아주 자랑스럽게도 한국인 조경현 박사님이 제안한 방법입니..

Gated Recurrent Units (GRU)

https://yjjo.tistory.com/18

Gated Recurrent Units (GRU)

Recommendations

////////