ReLU do not becoming gradient 0 when input becomes larger which can prevent the Vanishing Gradient problem.
Non-saturating nonlinearity
Creator
Creator
Seonglae ChoCreated
Created
2024 Sep 12 21:24Editor
Editor
Seonglae ChoEdited
Edited
2024 Sep 12 21:26Refs
Refs