In the context of this exercise, what does a validation set contain?
A validation set contains data that is used to determine how well a model is able to make predictions.
Generally, we first fit, or train, the model on training data. Once it has been trained, we validate how accurate it is by running the predictor with data from the validation set. After it makes the predictions, we check if they were correct based on the validation labels from the validation dataset. This process is applied to each point, and finally, the validation accuracy can be computed.
For an example, say that we have a predictor that determines what animal appears in an image. The validation set would consist of images with their labels, so that the model, once it has made predictions, can check how accurate it was.