Kaiming initialization

Creator
Creator
Seonglae Cho
Created
Created
2025 Jan 21 15:12
Editor
Edited
Edited
2025 Jan 21 15:16
Refs
Refs
ReLU
optimized since ReLU removes negative numbers while Kaiming initialization consistently preserves output distribution and so prevents reduction in output variance
WN(0,2NeuronCount)W \sim \mathbb{N}(0, \frac{2}{NeuronCount})
 
 
 
 
 
 
 
 

Recommendations