Weight Initialization

Creator

Creator

Created

Created

2023 Jun 6 8:40

Editor

Editor

Edited

Edited

2025 Mar 14 16:52

Refs

Refs

Parameter initialization

weight to small random numbers

bias (zero or small nonzero)

Weight Initialization Usages

He initialization

Xavier Initialization

Gaussian Initialization

Kaiming initialization

0025 Initialization - Deepest Documentation

0025 Initialization - Deepest Documentation

https://deepestdocs.readthedocs.io/en/latest/002_deep_learning_part_1/0025/

Backlinks

Mechanistic interpretability Transformer Training Random Transformer Meta Learning torch.manual_seed()Residual Connection AI Optimization

Recommendations

/////