machine learning

Question 1

machine learning

Answer 1

• finding patterns from data using algorithms
• exploiting them to carry out some task

Question 2

uses of machine learning

Answer 2

• detect or learn structures and relationships in data
• label or assign observations to different classes
• make predictions based on previously obtained knowledge

Question 3

Bayesian framework

Answer 3

• bayes theorem
• updates the probability of a hypothesis as more information becomes available

Question 4

inputs vs outputs

Answer 4

• inputs:
  
  • discrete
  • continuous
  • scalar or vector-valued
• outputs:
  
  • determined by the task to be performed

Question 5

machine learning tasks

Answer 5

• classification
  
  • output composed of discrete labels (classes)
  • identify class memberships of different inputs
• regression
  
  • where outputs are continuous random variables or ordered discrete variables
  • predict or estimate a response to a given input

Question 6

supervised vs unsupervised learning

Answer 6

• supervised:
  
  • data used to train an algorithm to predict the response of a new input
  • labelled data
• unsupervised:
  
  • looks for structure in the outputs (or inputs) without reference to inputs (or outputs)
  • discovers new knowledge
  • unlabelled data

Question 7

supervised learning

Answer 7

• relationship known between inputs and outputs:
  
  • y = f(x) + ε
  • ε = some error function
    
    • generally ignored in classification (=0)
• allows you to determine optimal function for further predictions about new data

Question 8

unsupervised learning

Answer 8

• finding new knowledge from data without training an algorithm using known input-output pairs
• similar inputs and outputs grouped to find a relationship
• density estimation problem
• useful when training data is rare and expensive (often)

Question 9

clustering

Answer 9

• example of unsupervised learning
• group together outputs into clusters
• define number of clusters C ad hoc or with related model
• estimate which cluster each data point belogns to
• assign attributes to points in a cluster e.g. ‘normal’ height
• cluster label given by z
  
  • hidden variable = inferred, not directly observed
• similarity of points defined by distance

Question 10

distance function

Answer 10

• d(x,y)
• must fulfil:
  
  • d(x,y) ≥ 0
  • d(x,y) = 0 iff x = y
  • d(x,y) = d(y,x) (symmetry)
  • d(x,y) ≤ d(x,z) + d(z,y) (triangular inequality)
    
    • any point in between increases distance

Question 11

distance

Answer 11

• different types
  
  • different properties and convenience levels
• generally euclidean distance

Question 12

types of clustering

Answer 12

• similarity-based
  
  • N points
  • input = N x N matrix of pairwise distances
    
    • relative distance between all pairs of points
  • gene expression, DNA sequences
• feature-based
  
  • each of N objects has M different features
  • input = N x M matrix
  • patient/control samples

Question 13

k-nearest-neighbour clustering

Answer 13

• non-parametric classification
• K = number of neighbours taken into consideration
• look for K nearest neighbours of input x to decide its state
  
  • if all neighbours same cluster designate to that cluster
  • if K > 1 and neighbour sin different clusters
    
    • assign probabilities and assign to highest
• assign all points and paint
• can also use K = 1

Question 14

hierarchical clustering

Answer 14

• allows clusters to be nested inside one another
• main method
• use clustering diagrams to create gene expression profiles
• produces binary trees/dendrograms

Question 15

agglomerative clustering

Answer 15

• type of hierarchical clustering
• N observations assigned to their own cluster C
• merge 2 most similar clusters
  
  • minimum distance
• create new cluster C_k until only 1 cluster remains
• cluster distance can be determined with:
  
  • average position of points in each cluster
  • position of closest point in each cluster

Question 16

divisive clustering

Answer 16

• another method of hierarchical clustering
  
  • faster than agglomerative
• assing all objects to one cluster and split
• use dissimilarity instead pf similarity to split
• harder to program
• but looks at all data from the beginning
  
  • agglomerative doesn’t - can mislead early on

Question 17

regression

Answer 17

• supervised learning
• use a model to describe continuous response
  
  • see how well data fits model
• need the right model
  
  • e.g. linear/non-linear
  • may look different to what it actually is so be wary

Question 18

process of regression

Answer 18

• model building
  
  • variable selection
  • relation of response to covariates
• data fitting
  
  • effect of each independent variable
  • confidence in the model’s description
  • statistical significance
• model checking
  
  • do assumptions hold
• model validation
  
  • does it work on new data

Question 19

statistical models

Answer 19

• components:
  
  • initial observed relationship between X and Y
    
    • before assumptions have been made about response distribution
  • error
    
    • random variable
    • independent
    • normally distributed

Question 20

multiple testing problem

Answer 20

• need to adjust significance/p value when testing many hypotheses
  
  • more tests means higher number of apparently signficant results that are due to chance
• bonferroni correction
  
  • individual hypotheses tested with p = α/m
    
    • α = overall desired p value
    • m = number of hypotheses

Question 21

problems in statistical learning theory

Answer 21

• multiple testing
• curse of dimensionality
  
  • inference more difficult as dimension increases
    
    • number of independent variables
• model selection
  
  • e.g. K in K-nearest
  • also overfitting
• no free lunch theorem
  
  • no universally best model
  • algorithms behave differently depending on context