Test error
The test error is the average error that results from using a statistical learning method to predict the response on a new observation, one that was not used in training the method.
Training Error
In contrast, the training error can be easily calculated by applying the statistical learning method to the observations used in its training.
Which error dramatically underestimates the other?
Training error to Test error
Training- versus Test-Set Performance

Validation-set approach

Drawback of Validation set approach

K-fold Cross-validation

K-fold Cross-validation in detail

The Bootstrap
