Gradient Descent

Creator
Creator
Alan JoAlan Jo
Created
Created
2021 Oct 6 10:3
Editor
Editor
Alan JoAlan Jo
Edited
Edited
2023 Sep 20 2:22
Refs
Refs

Steepest descent algorithm

notion image
Statistical Thinking
Update your probability not change (
Learning Rate
)
gradient ascent maximize and descent minimize function
Iteratively subtract the derivative of
Loss Function
wrt weight from the weight with the learning rate and then
Cost Function
decreases
Repeat until the weight convergence (
Local extremum point
)
Gradient Descents
 
 
Gradient Descent Notion
 
 
 
 

Dual number

 
 

Recommendations