5. Sampling methods

Question 1

Sampling techniques

Answer 1

• Validation (Stratified (partition) sampling)
  —- Cross validation
  —- Validation Set Approach (Random partitioning)
  —- K-fold Cross validation / Leave-1-out Cross validation
• Bootstrap (Sampling with replacement)
• Regularization (to prevent overfitting)
  —- Best subset selection
  —- Stepwise Selection ( Forward and Backward )

Question 2

Purpose of sampling

Answer 2

• Choosing the right model
• Obtain information about the test error
• Data reduction

( Quantitavily MSE, Qualitatively (AUROC or misclassification rate) )

Question 3

Test error vs. Training error

Answer 3

Test error: hard to estimate
Training error: easy to calculate