ANNs

Question 1

ANN Description

Answer 1

• A network of many simple processors (”units”).
• Units are connected by communication channels (”connections”).
• Units operate on their local data and inputs they receive via the connections.
• ANNs are distributed system, made up of simple processing units (”artificial neurons”).

Question 2

Neural Networks can be classified based on the following attributes:

Answer 2

• Learning Methods:*
• Supervised
• Unsupervised
• Topology/Architecture:*
• Single-layer
• Multi-layer
• Self-organised
• Connection Type:*
• Feedforward
• Recurrent

Question 3

ANN’s pros and cons:

Answer 3

Pros:

• Flexible inputs types and number
• Resistant to errors in the training data
• Manage non-linear dynamics
• Fast evaluation of performance

Cons:

• Long training time/computationally intensive
• Overfitting risk
• Difficult interpretation

Question 4

Single Artificial Unit Structure

Answer 4

Input
Weights 
Input function
Activation function
Output

Question 5

Perceptron

Answer 5

Simplest type of neural network: feed forward with one single unit.

Logic:

1. Takes all inputs and all weights in a linear combination (Input function).
2. Output is calculated by step activation function (@ theta).
3. Learning occurs through the “Perceptron Learning Rule”.
4. Learning rule is applied at each observation until convergence of modified weights or other stopping rule.
5. One pass-through for the whole training set is called an epoch of training.
6. Many epochs are necessary for convergence: training set error ceases to improve.

Question 6

Bias

Answer 6

The threshold $\theta $ is important. Usually you need to adjust its level to to obtain better results and quick convergence time. This can be done by introducing the concept of Bias → just move the threshold to the left side of the activation function and train it as a weight.

Question 7

Type of activation functions

Answer 7

• Sigmoid logistic
• Sigmoid Hyperbolic Tangent function
• Rectified Linear Unit (ReLU)