In the context of this exercise, what is a step?
A step is essentially an update of parameter values, with the aim of decreasing the loss and moving in the direction of the local minimum.
Steps do not only have to increase the parameter value but can also decrease its value. A “step size” determines how big of a change we make to the parameters each step.
For example, as illustrated in this animation from exercise 5, the update of the b value from
12 can be considered a step.