Gradient Descent with Momentum

We often think of optimization with momentum as a ball rolling down a hill. This isn't wrong, but there is much more to the story.