Why Gradient Descent Zigzags and How Momentum Fixes It
PLOT_STEPS = 55 x_ = np.linspace(-5, 5, 500) y_ = np.linspace(-2.2, 2.2, 500) X, Y = np.meshgrid(x_, y_) Z = loss(X, Y) fig = plt.figure(figsize=(16, 10), facecolor=”#FAFAF8″) gs = GridSpec(2, 3, figure=fig, hspace=0.45, wspace=0.38, left=0.07, right=0.97, top=0.88, bottom=0.08) COLORS = { “gd”: “#E05C4B”, “mom_good”: “#3A7CA5”, “mom_large”: “#F4A536”, “contour”: “#D4C9B8”, “minima”: “#2A9D5C”, “start”: “#444444”, } PANEL_TITLES…
