Seong-lae ChoArtificial Neural Network
A statistical learning algorithm inspired by biological neural networks in machine learning and cognitive science
Narrowly defined AI
Neural Network is a differentiable computer.
Data Representation + Data Classification + Algorithm
Neural network layer common architecture
- layer normalization
- main logic like Attention Mechanism or mlp with activation function
- projection to original size (even if it is same dimension size)
- residual connection
Neural Network Notion
Visualization
Tensorflow — Neural Network Playground
Tinker with a real neural network right here in your browser.
https://playground.tensorflow.org/
Zero to Hero
Neural Networks: Zero To Hero
A course by Andrej Karpathy on building neural networks, from scratch, in code. We start with the basics of backpropagation and build up to modern deep neural networks, like GPT. In my opinion language models are an excellent place to learn deep learning, even if your intention is to eventually go to other areas like computer vision because most of what you learn will be immediately transferable.
https://karpathy.ai/zero-to-hero.html