Decision Trees

Question 1

Equation for Entropy

Answer 1

SUM(
-P(outcomeN)*log2(outcomeN)
)

Question 2

Equation for Information Gain

Answer 2

Entropy(S) -
Sum
(
P(Rows with Feature=Value in Set)
*
Entropy(Feature=Value)
)

Question 3

What are the two type of Decision Trees?

Answer 3

Classification(Target is discrete)
Regression(Target is continuous)

Question 4

What are the stopping criteria for decision trees?

Answer 4

1) All or nearly all data has same class label
2) When there are no features to further split upon
3) When tree reaches a predefined depth

Question 5

What are feature selection measures?

Answer 5

deciding which features are to be considered as the root node and at each level.
Eg: Gini Index, Information Gain