Texonom
Texonom
/
Engineering
Engineering
/Data Engineering/Artificial Intelligence/Machine Learning/Neural Network/Neural Network Structure/Activation Function/
ReLU
Search

ReLU

Creator
Creator
Seonglae Cho
Created
Created
2023 Mar 7 13:5
Editor
Editor
Seonglae Cho
Edited
Edited
2024 Sep 30 15:10
Refs
Refs

Rectified Linear Unit

max(0,z)max(0, z)max(0,z)
or max(0,xW,b)max(0, xW, b)max(0,xW,b)
ReLU Variants
PReLU
Leaky ReLU
Dying ReLU
 
 
 
 
 
 
 

Backlinks

Non-saturating nonlinearityNeuron SAELoss FunctionWeight InitializationKaiming initializationVanishing Gradient

Recommendations

Texonom
Texonom
/
Engineering
Engineering
/Data Engineering/Artificial Intelligence/Machine Learning/Neural Network/Neural Network Structure/Activation Function/
ReLU
Copyright Seonglae Cho