1. 9
  1. 3

    Interesting. I’m taking Nonlinear Optimization (taught by Steve Wright) and we give some theoretical analysis of these methods, but it’s nice to see the big picture with visualization.

    1. 2

      Great. Possible to share the link to the notes or video lectures (if any). I have gone through vanilla grad. descent, SGD and Mini-bathGD, and currently going through the Stochastic Average Gradient solver. Would love to dig deep into the topic.