Seong-lae ChoConvolutional Neural Network
A Convolutional Neural Network (CNN) is a neural network optimization technique based on the assumption that nearby data points have strongly dependent features, rather than being iid (independent and identically distributed). CNNs assume that each pixel depends on its neighbors, significantly reducing the number of parameters through convolution operations. Note that CNNs actually use Correlation rather than Convolution in the mathematical sense.
The architecture alternates between Convolutional Layers and Pooling Layers to extract features from input images, then performs classification through Fully Connected (FC) Layers.
Hyperparameters
- Convolution layer's filter type, size, and stride, which determine each activation map
- Placements and types of convolution layers, pooling layers, and activation layers
Typical Architecture
The standard CNN architecture follows this pattern:
Where:
- N is usually up to ~5 (number of convolution-activation pairs before pooling)
- M is large (number of convolution-pooling block repetitions)
- 0 ≤ K ≤ 2 (number of fully connected layers before output)
However, recent advances have challenged this traditional paradigm.
CNN Notion
CNN Models