Boosting

Question 1

Adaboost

Answer 1

• Forest of trees (stumps, 1 root, 2 leaves)
  
  • stump: week learner (this is what we want)
  • stumps get weights
  • stumps influences each other (1. influence 2., 2. influence 3.). each stump is created by taking mistakes of previous into account
• how to creat ada boost
  
  • samples get weighted
  • make first stump (lowest gini index)
  • weight stump: calc form total error: sum of weights of incorrecly classified samples
  • updating weights: increase weights of incorrectly classified samples (weighted by stump weighst), decrease sample weights of correctly classified samples
  • normalize weights: sum of weights = 1
  • second stump: calc weighted gini index or new data set with duplicate copies of samples with largest sample weights, then give them all same sample weights
• How to make classifications
  
  • sum amounts of say for group of stumps that say 1 or 0
• *

Question 2

Hard margin / soft margin

Answer 2

• hard margin: sensitive to noise
• soft margin: mistrust bad points