Rectified Linear Unitmax(0,z)max(0, z)max(0,z)or max(0,xW,b)max(0, xW, b)max(0,xW,b)ReLU VariantsPReLULeaky ReLUDying ReLU