Seonglae ChoSteepest descent algorithm
Gradient based learning can be easily continued
gradient ascent maximize and descent minimize function
Iteratively subtract the derivative of Loss Function wrt weight from the weight with the learning rate and then Cost Function decreases
Repeat until the weight convergence (Local extremum point)
Gradient Descent를 시각적으로 이해하려면 Loss Function 우리가 각 데이터에 대해 loss function을 3차원에서 가지고 있고 gravity처럼 아래로 내려가는 것이다. 최대한 다양한 데이터를 활용해 모든 data point에서 loss 0에 가까워지도록 하는 것
Gradient Descents
Gradient Descent Notion
Learning Long-term dependency with Gradient Descent is difficult
0021 Gradient Descent & Momentum - Deepest Documentation
https://deepestdocs.readthedocs.io/en/latest/002_deep_learning_part_1/0021/
Dual number
adventures/dual-numbers-and-gradient-descent.ipynb at master · samuelbelko/adventures
Simple math oriented projects using Julia and JupyterLab. - adventures/dual-numbers-and-gradient-descent.ipynb at master · samuelbelko/adventures
https://github.com/samuelbelko/adventures/blob/master/dual-numbers-and-gradient/dual-numbers-and-gradient-descent.ipynb
Online gradient descent written in SQL • Max Halford
Edit: this post generated a few insightful comments on Hacker News. I’ve also put the code in a notebook for ease of use. Introduction Modern MLOps is complex because it involves too many components. You need a message bus, a stream processing engine, an API, a model store, a feature store, a monitoring service, etc. Sadly, containerisation software and the unbundling trend have encouraged an appetite for complexity. I believe MLOps shouldn’t be this complex.
https://maxhalford.github.io/blog/ogd-in-sql/