Batch Normalization

Creator

Creator

Seonglae Cho

Created

Created

2023 Mar 7 12:59

Editor

Editor

Seonglae Cho

Edited

Edited

2024 Feb 29 6:22

Refs

Refs

Across the batch dimension, any neuron to have unit gaussian distribution

레이어마다 Normalization을 하는 레이어를 두어, 변형된 분포가 나오지 않도록 하는 것

Layer Normalization 에서 차원 하나만 바꾸면 된다

Limitation

dependent to mini batch size

hard to apply to RNN

Batch Normalization Methods

Internal Covariate Shift

배치 정규화(Batch Normalization)

gaussian37's blog

배치 정규화(Batch Normalization)

https://gaussian37.github.io/dl-concept-batchnorm/

배치 정규화(Batch Normalization)

Batch normalization

Batch normalization is a method used to make training of artificial neural networks faster and more stable through normalization of the layers' inputs by re-centering and re-scaling. It was proposed by Sergey Ioffe and Christian Szegedy in 2015.

Batch normalization

https://en.wikipedia.org/wiki/Batch_normalization

Batch normalization

Recommendations

////////