In the context of this exercise introducing gradient descent, is gradient descent only applicable to two dimensions?
No, gradient descent is not limited to two dimensions.
For the models we use in the lesson, we apply it in two dimensions, but it can also apply to three, four, or even an infinite number of dimensions. This is because, in general, gradient descent is used to find the minimum of a function, regardless of how many dimensions it is in.
For an example of gradient descent applied in more than two dimensions, we can picture gradient descent applying to three dimensions, where instead of a curve in two dimensions, we have a sort of topology with hills and valleys. In this case, gradient descent will start at some point on this surface and attempt to find a minimum, going down the hill into a valley or low point.