How can we determine when loss changes very slowly?


#1

Question

In the context of this exercise, how can we determine when loss changes very slowly?

Answer

We learned that convergence is reached when loss stops completely or changes very slowly.

We can determine when the loss is changing very slowly in a few ways.

One way, which is used in this exercise, is by looking at a graph showing the number of iterations vs the slope or intercept value. When we see that the parameter value is changing very slowly, we can choose one of the values around which that happens.

Another way is to choose our own loss value, such that once the loss is less than that value, we can say there is convergence. For example, we can choose a value like 0.01, and if the loss is less than this, then it has converged and we can stop.