Neural-network Algorithms.

Question 1

What is an artificial neural network inspired by?

Answer 1

The brain’s neurons

An artificial neural network carries out pattern matching in a manner similar to how neurons in the brain operate.

Question 2

A classical artificial neural network consists of which layers?

Answer 2

• Input layer
• Hidden layers
• Output layer

Each layer serves a specific function in processing data.

Question 3

What is a node in an artificial neural network sometimes referred to as?

Answer 3

Perceptron

A perceptron is analogous to a neuron in the network.

Question 4

What is the process called when data flows through each layer in order from input to output?

Answer 4

Feed-forward

This process is essential for the functioning of neural networks.

Question 5

In the example of deciding whether to go skiing, what are the inputs considered?

Answer 5

• Has it snowed?
• Are the slopes busy?
• Is the avalanche risk high?

These inputs are evaluated to make a decision.

Question 6

How are decisions weighted in the skiing example?

Answer 6

• Snow: 5
• Crowds: 2
• Avalanche risk: 4

The weights reflect the importance of each factor in the decision-making process.

Question 7

What function decides whether to activate a node based on a threshold value?

Answer 7

Activation function

This function plays a crucial role in determining the output of the node.

Question 8

What type of learning uses a labeled dataset to train the model?

Answer 8

Supervised learning

This method involves using known outcomes to guide the training process.

Question 9

What is the purpose of a bias value in neural networks?

Answer 9

To reduce error from the output

The bias helps optimize the weights to improve accuracy.

Question 10

What is a convolutional neural network used for?

Answer 10

Image or time-series-related AI systems

This type of network is a modern update to artificial neural networks.

Question 11

What do convolution layers in a convolutional neural network do?

Answer 11

Look for high-level features based on low-level features

This process helps in feature extraction from images.

Question 12

What does deep learning refer to?

Answer 12

Many layers of nodes

This approach allows for more complex models but is computationally intensive.

Question 13

What technology allows for complex neural networks to be processed efficiently?

Answer 13

Graphics Processing Units (GPUs)

GPUs enable the execution of tasks requiring significant computational power.

Question 14

What is the basic structure of an artificial neuron, or perceptron?

Answer 14

• Receives input
• Processes input
• Produces output

The perceptron mimics the behavior of biological neurons using input signals, weights, and an activation function.

Question 15

True or false: The single-layer perceptron can solve non-linear problems effectively.

Answer 15

FALSE

Minsky and Papert demonstrated that single-layer perceptrons could not solve basic logical tasks like XOR.

Question 16

What are the key differences between logistic regression and a 1-neuron perceptron?

Answer 16

• Logistic regression uses the logistic function
• A perceptron typically uses a step function
• Logistic regression outputs probabilities
• A perceptron produces hard binary outputs

These differences highlight the perceptron’s decisiveness but less flexibility in representing probabilities.

Question 17

What is the Hebbian learning rule?

Answer 17

Cells that fire together wire together

This rule strengthens connections between neurons that are activated simultaneously.

Question 18

What is a training epoch in the context of neural networks?

Answer 18

A single pass through the entire training dataset

After each epoch, the network’s weights are updated based on the error calculated for each data point.

Question 19

What is the purpose of a training stop rule?

Answer 19

• Determine when to end training
• Prevent overfitting or underfitting

Common stop rules include fixed number of epochs and early stopping.

Question 20

What does the activation function do in a neuron?

Answer 20

Produces the output based on the weighted sum of inputs

It introduces non-linearity into the network, allowing it to model complex patterns.

Question 21

List common types of activation functions.

Answer 21

• Sigmoid
• ReLU (Rectified Linear Unit)
• Tan h (Hyperbolic Tangent)
• Softmax

Each activation function has different properties and is chosen based on the model’s needs.

Question 22

What is the role of a weight matrix in a neural network?

Answer 22

Contains the weights applied to the inputs at each layer

The weight matrix reflects the differences between classes and helps in classification.

Question 23

What is backpropagation?

Answer 23

A supervised learning algorithm that adjusts weights by propagating error backwards

It allows the training of deep neural networks and is effective in minimising error.

Question 24

What is the Adam optimisation algorithm known for?

Answer 24

• Handling sparse gradients
• Adapting the learning rate during training
• Computational efficiency

Adam is widely used in deep learning due to its effectiveness with large amounts of data.

Question 25

What is the limitation of **Hebbian learning**?

Answer 25

* Inability to solve non-linear problems * No error correction mechanism ## Footnote This makes it challenging to train multilayer networks effectively.

Question 26

What does a **classifier** do in the context of neural networks?

Answer 26

Assigns a label to an input data point based on features ## Footnote It categorises input data into one or more classes.

Question 27

What is the significance of the **1-neuron perceptron** in artificial neural networks?

Answer 27

Serves as a basic building block for more complex networks ## Footnote It provides insights into how more complex neural networks operate and learn.

Question 28

What is the foundational role of the **1-neuron perceptron model** in artificial neural networks?

Answer 28

It serves as a basic building block, providing insights into how more complex neural networks operate and learn. ## Footnote The perceptron’s ability to classify data based on a linear decision boundary made it a pioneering model in the field of AI.

Question 29

What is a significant limitation of the **1-neuron perceptron**?

Answer 29

It struggles with non-linear problems or more complex data patterns. ## Footnote These limitations have driven the development of more advanced neural network architectures.

Question 30

What type of neural network is developed due to the limitations of the **1-neuron perceptron**?

Answer 30

Multilayer perceptrons (MLP) ## Footnote MLP consists of multiple layers of neurons, each capable of learning more complex patterns.

Question 31

What do **non-linear activation functions** in MLP allow these networks to do?

Answer 31

Overcome the limitations of simple perceptrons and model complex, non-linear relationships. ## Footnote This advancement enables MLP to perform tasks previously impossible with single-layer perceptrons.

Question 32

List some applications of **multilayer perceptrons (MLP)**.

Answer 32

* Image recognition * Natural language processing ## Footnote MLP can solve a wide range of problems due to their ability to learn complex patterns.

Question 33

True or false: The **1-neuron perceptron** can outperform traditional methods like logistic regression.

Answer 33

TRUE ## Footnote Understanding the foundational concepts of perceptrons equips learners to tackle modern AI challenges.

Question 34

What is the ongoing process related to **learning rules** in artificial intelligence?

Answer 34

Improving learning rules is an ongoing, continuous process. ## Footnote Many specific learning and optimizing rules are effective in different tasks.

Question 35

What is the **Multilayer Perceptron (MLP)**?

Answer 35

A type of artificial neural network organized into layers with feedforward connections ## Footnote MLP consists of an input layer, one or more hidden layers, and an output layer.

Question 36

In an MLP, how is information processed?

Answer 36

Information flows in one direction—from the input layer through hidden layers to the output layer ## Footnote Each neuron in a layer is connected to every neuron in the subsequent layer.

Question 37

What is the **universal approximation theorem**?

Answer 37

An MLP with two hidden layers can approximate any algebraic function ## Footnote This highlights the power of MLP in solving a wide range of problems.

Question 38

What is a common rule of thumb when designing an MLP?

Answer 38

Start with a large number of neurons in the first hidden layer, reducing by half in each subsequent layer ## Footnote This helps refine the understanding of input data.

Question 39

What is an **artificial neuron**?

Answer 39

A computational unit that processes inputs using a weighted sum and an activation function ## Footnote Neurons are inspired by biological neurons in the brain.

Question 40

What do **feedforward connections** refer to?

Answer 40

The flow of data through layers in a single direction—forward ## Footnote There are no cycles or loops in a feedforward neural network.

Question 41

What are **Recurrent Neural Networks (RNN)** designed for?

Answer 41

Handling sequences of data with feedback loops ## Footnote RNNs retain information across time steps, making them ideal for tasks like time series forecasting.

Question 42

What is a key application of RNNs?

Answer 42

Text generation ## Footnote RNNs can learn to generate new sentences that mimic the style and structure of training data.

Question 43

What are **Convolutional Neural Networks (CNNs)** tailored for?

Answer 43

Processing data with a grid-like topology, such as images ## Footnote CNNs detect local patterns like edges and textures in images.

Question 44

What is a **receptive field** in CNNs?

Answer 44

A small region of the input layer to which each neuron is connected ## Footnote This design makes CNNs efficient for tasks like image classification.

Question 45

What is one of the most famous applications of MLP?

Answer 45

Optical character recognition (OCR) for handwritten numbers ## Footnote MLP was crucial for automating the sorting of mail in post offices.

Question 46

How were digit images processed for MLP in OCR?

Answer 46

Converted into a 28x28 pixel grid, flattened into a 784-dimensional vector ## Footnote Each pixel was assigned a value based on its intensity.

Question 47

What do the weight matrices in hidden layers of MLP represent?

Answer 47

Filters that detect specific features of the digits ## Footnote These filters can reveal insights into how the network processes input data.

Question 48

What is the efficiency of MLP in recognizing handwritten digits?

Answer 48

Achieving accuracy rates well above 90% ## Footnote The trained network generalizes well to new images of digits.

Question 49

What do hidden layers in MLP develop for natural image recognition?

Answer 49

Filters that represent basic geometric shapes like lines and curves ## Footnote These filters are common in many natural images.

Question 50

What is the significance of **style transfer** in CNNs?

Answer 50

Applying the style of one image to another while retaining its content ## Footnote This technique can also be applied to text generation and MLP tasks.

Question 51

What does the evolution from a simple 1-neuron perceptron to complex ANN types showcase?

Answer 51

The potential of neural networks in solving diverse problems ## Footnote This includes the introduction of layering and specialized architectures for sequences and spatial data.

Question 52

What is the **purpose** of a **clustering approach** in unsupervised learning?

Answer 52

To group unlabelled data to reveal hidden structures ## Footnote Clustering helps uncover meaningful groups or patterns in data without predefined labels.

Question 53

The origins of **unsupervised learning** can be traced back to which decade?

Answer 53

1980s ## Footnote This period saw the development of Kohonen networks and self-organising maps.

Question 54

What are the **two main capabilities** of **self-organising maps (SOM)**?

Answer 54

* Clustering * Feature reduction ## Footnote SOM allows high-dimensional data to be visualised and analysed in a more manageable form.

Question 55

Name the **four modern clustering techniques** mentioned.

Answer 55

* K-Means * Hierarchical Clustering * DBSCAN * Gaussian mixtures ## Footnote These techniques offer robust and scalable solutions for organising complex datasets.

Question 56

What is the primary goal of the **K-Means algorithm**?

Answer 56

To partition data into 𝑘 clusters by minimising the within-cluster variance ## Footnote K-Means is praised for its simplicity and efficiency, especially with large datasets.

Question 57

What are the **steps** involved in the **K-Means algorithm**?

Answer 57

* Initialisation: Randomly assign 𝑘 initial cluster centroids * Assignment: Assign each data point to the nearest centroid * Update: Recalculate the centroids * Repeat: Iterate until centroids no longer change significantly ## Footnote K-Means is sensitive to the initial placement of centroids and assumes spherical clusters.

Question 58

True or false: **Hierarchical Clustering** requires the number of clusters to be predefined.

Answer 58

FALSE ## Footnote Hierarchical Clustering builds a tree of clusters (dendrogram) without predefining the number of clusters.

Question 59

What are the two approaches of **Hierarchical Clustering**?

Answer 59

* Agglomerative (bottom-up) * Divisive (top-down) ## Footnote Agglomerative starts with each data point as its own cluster, while divisive starts with all data points in one cluster.

Question 60

What does **DBSCAN** stand for?

Answer 60

Density-Based Spatial Clustering of Applications with Noise ## Footnote DBSCAN is effective in identifying clusters of varying shapes and sizes.

Question 61

What are the two parameters required by **DBSCAN**?

Answer 61

* Maximum distance between two points to be considered neighbours * Minimum number of points required to form a dense region ## Footnote DBSCAN is robust to noise and can handle clusters of different shapes.

Question 62

What is the main objective of **Principal Component Analysis (PCA)**?

Answer 62

To reduce the dimensionality of the data while retaining key information ## Footnote PCA identifies directions along which the data varies the most and projects the data onto these directions.

Question 63

How does **PCA** differ from clustering methods?

Answer 63

* Objective: PCA reduces dimensionality; clustering groups data points * Output: PCA transforms data; clustering assigns data points to clusters * Interpretability: PCA simplifies data; clustering identifies natural groupings ## Footnote PCA is often used as a preprocessing step before clustering.

Question 64

What is the **Elbow Method** used for?

Answer 64

To determine the optimal number of clusters ## Footnote It involves plotting the within-cluster sum of squares (WCSS) against the number of clusters.

Question 65

What does the **Silhouette Score** measure?

Answer 65

How similar each data point is to its own cluster compared to other clusters ## Footnote A higher score indicates better clustering.

Question 66

What is a common approach to **summarise clusters**?

Answer 66

Summarise based on the features of the data points within each cluster ## Footnote For example, average age, income, or spending habits in customer segmentation.

Question 67

In what scenarios is **clustering** often used?

Answer 67

* Customer segmentation in marketing * Grouping countries based on economic indicators * Identifying disease subtypes in medical datasets ## Footnote Clustering helps tailor strategies and identify trends.