Ridge Regression uses a L2 update: \[\theta _{next} = \theta _{current} - currentstepsize \cdot (\theta _{current}^{'}+\gamma \cdot \theta _{current})\] \[currentstepsize = stepsize\cdot \frac{1}{\sqrt{iterIndex})}\]