Unlike sign function for Back Propagation, use non zero derivative
Mimics Neuron’s activation for non-linearity
Activation values are also known as Feature Maps in a neural network
- sigmoid function - probability
- sign function - predict class
Activation Functions
Activation Function Notion