Is gradient descent only applicable to two dimensions?

Question

In the context of this exercise introducing gradient descent, is gradient descent only applicable to two dimensions?

Answer

No, gradient descent is not limited to two dimensions.

For the models we use in the lesson, we apply it in two dimensions, but it can also apply to three, four, or even an infinite number of dimensions. This is because, in general, gradient descent is used to find the minimum of a function, regardless of how many dimensions it is in.

For an example of gradient descent applied in more than two dimensions, we can picture gradient descent applying to three dimensions, where instead of a curve in two dimensions, we have a sort of topology with hills and valleys. In this case, gradient descent will start at some point on this surface and attempt to find a minimum, going down the hill into a valley or low point.

13 Likes

But what could be the equation for that? Or is each time we increase a dimension the equation differ?

5 Likes

dont have explaination for this -2/N

2 Likes

N is for normalizing, and 2 comes when we differentiate the squared error term. If you go through the derivation, you can understand it.

1 Like

you can find some info in this video: https://www.youtube.com/watch?v=IHZwWFHWa-w
in Calculus we have the gradient vector. This gradient give us the direction in wich your function (n variable function) grows faster. If you multiply it by -1 then you will find the oposite direction (fastest decresing).

Hope that help you.

3 Likes

Do you have any book title that covers this topics with math in an engeneering level?

1 Like