While this is pretty cool, and probably a good way to visualize gradient descent, where gradient descent really shines is when there isn’t a simple closed-form answer to the question you’re trying to answer. You don’t need 2000 iterations of iterative optimization to do least-squares regression!
Just went through all this in the Coursera course mentioned. This was a much more succinct way of explaining it, great writeup.
While this is pretty cool, and probably a good way to visualize gradient descent, where gradient descent really shines is when there isn’t a simple closed-form answer to the question you’re trying to answer. You don’t need 2000 iterations of iterative optimization to do least-squares regression!