Lesson 3 - Supervised learning

Question 1

Supervised Learning formalization

Answer 1

• Input: x∈X, X is the space of instances of the task
• Output: y∈Y, changes depending on the task:
  
  • classification, binary
  • multi-class classification, natural numbers
  • regression, real numbers
• we assume that an oracle exists
• the training data is composed from historical records (x, y), generated by the oracle
• an hypothesis is selected from an hypothesis space using training data

Question 2

Types of oracle

Answer 2

• Oracle deterministic
  
  • a function associates input and output
  • target function is deterministic and unknown
• Oracle stochastic
  
  • input and output chosen according to a certain probability distribution
  • target function is stochastic and unknown

Question 3

Training set

Answer 3

A series of pairs, generated according to a probability function

Question 4

Empirical error/risk

Answer 4

• error on training data

Question 5

Ideal error/risk

Answer 5

• expected error on a given pairs x, y drawn according to the probability distribution

Question 6

Characteristics of the selected hypothesis

Answer 6

• a plausible hypothesis is selected using training data
• it should generalize well
  
  • correct predictions also for unseen examples

Question 7

Inductive bias

Answer 7

• assumptions made on hypothesis space and learning algorithm
  
  • hypothesis space cannot contaion all possible functions
  • how the space is explored