Hebbian learning

Question 1

Hebbian learning inspiration

Answer 1

This learning method is based on Hebb’s rule - what fires together, wires together. Hebian learning can be used as an online PCA with adaptive tracking of the direction of largest variance.

Question 2

Problem with pure Hebbian Learning

Answer 2

1. Since the covariance matrix C is positive semi-definite, this update rule leads to an exploding weight vector. Thus we need to normalise the weight vector in every step
   2.

Question 3

HL normalisation methods

Answer 3

1. Explicit normalisation - divide the weight by its absolute value
2. Implicit normalisation - Oja’s Rule (normalized the vector length to 1 by introducing a decay term)

Question 4

Anti-Hebbian Rule

Answer 4

A novelty filter projects the data onto the smallest PC. The largest output for unexpected data is measured as an unexpected/novel data point. It is based on the Anti-Hebbian learning rule.