Image classification

Question 1

Name some properties of good features in images

Answer 1

1. Distinct and discriminative
2. Local
3. Invariant to translations and transformations
4. Invariant to brightness change
5. Efficient to compute
6. Robust to noise/ blurring

Question 2

How can the pixel values of a patch be calculated using the integral image

Answer 2

Bottom right corner - bottom left - top right + top left

Question 3

What is the idea behind ensemble learning.

Answer 3

Aggregate the results of several predictors into one prediction

Question 4

In what case will ensamble learning not lead to improved results over one predictor.

Answer 4

If all predictors are higly correlated, so they give the same output.

Question 5

What kind of error do sequential parallel learners reduce?

Answer 5

Sequential learners reduce mostly bias, while parallel reduce variance.

Question 6

Name 4 different ensamble methods, and which of them are parallel/ sequential.

Answer 6

Parallel:
Bagging ( Bootstrap aggregation)
Voting
Random forrests

Sequential:
Boosting

Question 7

How does Bagging work?

Answer 7

Create N versions of the training set, using sampling with replacement and train a weak learner on each one. Use Averaging for regression, and majority voting for classification.

Question 8

What is Out-Of-Bag error for bagging?

Answer 8

For each training sample, xi:

1. Find the predicition, y_hat_i, of all classifiers not containing xi in the training set.
2. Average y_hat_i
3. repeat and average for all training samples.

Question 9

What is boosting?

Answer 9

We compute classifiers sequential, by increasing the weight of missclassified examples each run.

Question 10

Describe the adaboost algorithm

Answer 10

1. Initialize all weights equally
2. Training subsets are bootstrapped from the full dataset using weighted sampling
3. Fit a classifier to the new set
4. Up-weigh missclassified weights, down-weigh the rest and repeat

Question 11

How can we calculate the probability that a decision tree prediction is correct?

Answer 11

We know how many samples from each class of the test set each node(prediction) contained.

Question 12

Describe the decision tree optimization for creating a feature space partition

Answer 12

At each node Sj
  for each feature
    for each value of this feature:
      evaulate I(Sj, Aj)
 chose the best feature and value for splitting
reapeat

Question 13

What are the two choices for categorizing tree optimization cost functions, I(Sj, Aj)?

Answer 13

1. Information Gain

2. Gini Index.

Question 14

Desribe the information gain cost function, I(Sj, Aj)

Answer 14

I(Sj, Aj) = Entropy parent - weighted average entropy of children.
Entropy = - sum p(xj) log(p(xj))

Question 15

Describe the Gini Index

Answer 15

Indicates how mixed classes are, perfect seperation results in score 0, 50/50 seperation results in score 0.5.

Gini = 1 - sum (p(yk))**2 
Final = weighted average of the Ginis

Question 16

What loss do we normally use for regression trees?

Answer 16

weighted average of MSE

Question 17

What can we do to prevent overfitting in decision trees?

Answer 17

Combine them to an esamble (Forests).

Question 18

For random forrests, how does the:
1. Number of features selected per node
2. Number of trees
3. Max depth
Affect bias and variance?

Answer 18

1. More features will increase variance, but might reduce bias.
2. More trees will reduce variance
3. Deeper trees will increase variance, but might reduce bias

Question 19

What is the difference between a random forrest and a boosting algorithm with decision trees as weak learners?

Answer 19

The trees differ as the splitting is randomized.

Question 20

Name some advantages of decision trees and dissadvantages of decissions trees?

Answer 20

Advantages:

1. Explainable model
2. Can handle multi class problems
3. Can handle categorical and continous variables
4. Requires little preprossesing
5. The cost is logarithmic in the number of samples

Disadvantages:

1. Prone to overfitting
2. Biassed towards classes with more datapoints