MLP
The rows of the weight matrix before the activation function can be thought of as directions in the embedding space, and that means activation of each neuron tells you how much a given vector aligns with some specific direction. The columns of the weight matrix after the activation function tell you what will be added to the result if that neuron is active.
Can have non-linear decision boundary
using 3 Perceptron, now available to separate XOR
Two layer
convex open or closed region
each line means perceptron
Three layer
arbitrary (complexity limited by number of neurons)
Hidden layer
except final layer