What are the activation functions?
Activation function translates the inputs into outputs. Activation function decides whether a neuron should be activated or not by calculating the weighted sum and further adding bias with it. The purpose of the activation function is to introduce non-linearity into the output of a neuron.
There can be many Activation functions like:
- Linear or Identity
- Unit or Binary Step
- Sigmoid or Logistic
- Tanh
- ReLU
- Softmax
Differentiate between AI, Machine Learning and Deep Learning.
Artificial Intelligence is a technique that enables machines to mimic human behavior.
Machine Learning is a subset of AI technique which uses statistical methods to enable machines to improve with experience.
Deep learning is a subset of ML which make the computation of multi-layer neural network feasible. It uses Neural networks to simulate human-like decision making.
In terms of Dimensionality Reduction, How does Autoencoder differ from PCAs?
- An autoencoder can learn non-linear transformations with a non-linear activation function and multiple layers.
- It doesn’t have to learn dense layers. It can use convolutional layers to learn which is better for video, image and series data.
- It is more efficient to learn several layers with an autoencoder rather than learn one huge transformation with PCA.
- An autoencoder provides a representation of each layer as the output.
- It can make use of pre-trained layers from another model to apply transfer learning to enhance the encoder/decoder.
Explain the architecture of an Autoencoder.
- Encoder: This part of the network compresses the input into a latent space representation. The encoder layer encodes the input image as a compressed representation in a reduced dimension. The compressed image is the distorted version of the original image.
- Code: This part of the network represents the compressed input which is fed to the decoder.
- Decoder: This layer decodes the encoded image back to the original dimension. The decoded image is a lossy reconstruction of the original image and it is reconstructed from the latent space representation.
Explain Autoencoders and it’s uses.
An autoencoder neural network is an Unsupervised Machine learning algorithm that applies backpropagation, setting the target values to be equal to the inputs. Autoencoders are used to reduce the size of our inputs into a smaller representation. If anyone needs the original data, they can reconstruct it from the compressed data.
What are Deep Autoencoders?
The extension of the simple Autoencoder is the Deep Autoencoder. The first layer of the Deep Autoencoder is used for first-order features in the raw input. The second layer is used for second-order features corresponding to patterns in the appearance of first-order features. Deeper layers of the Deep Autoencoder tend to learn even higher-order features.
A deep autoencoder is composed of two, symmetrical deep-belief networks:
- First four or five shallow layers representing the encoding half of the net.
- The second set of four or five layers that make up the decoding half.
Give some real-life examples where autoencoders can be applied.
- Image Coloring: Autoencoders are used for converting any black and white picture into a colored image. Depending on what is in the picture, it is possible to tell what the color should be.
- Feature variation: It extracts only the required features of an image and generates the output by removing any noise or unnecessary interruption.
- Dimensionality Reduction: The reconstructed image is the same as our input but with reduced dimensions. It helps in providing a similar image with a reduced pixel value.
- Denoising Image: The input seen by the autoencoder is not the raw input but a stochastically corrupted version. A denoising autoencoder is thus trained to reconstruct the original input from the noisy version.
what are the different layers of Autoencoders?
An Autoencoder consist of three layers:
- Encoder
- Code
- Decoder
Is there any variation of Autoencoders?
- Convolution Autoencoders
- Sparse Autoencoders
- Deep Autoencoders
- Contractive Autoencoders
What is a Bottleneck in autoencoder and why is it used?
The layer between the encoder and decoder, ie. the code is also known as Bottleneck. This is a well-designed approach to decide which aspects of observed data are relevant information and what aspects can be discarded.
It does this by balancing two criteria:
- Compactness of representation, measured as the compressibility.
- It retains some behaviourally relevant variables from the input.
What are capsules in Capsule Neural Network?
Capsules are a vector specifying the features of the object and its likelihood. These features can be any of the instantiation parameters like position, size, orientation, deformation, velocity, hue, texture and much more.
A capsule can also specify its attributes like angle and size so that it can represent the same generic information. Now, just like a neural network has layers of neurons, a capsule network can have layers of capsules.
What is a CNN?
Convolutional neural network (CNN, or ConvNet) is a class of deep neural networks, most commonly applied to analyzing visual imagery. Unlike neural networks, where the input is a vector, here the input is a multi-channeled image. CNNs use a variation of multilayer perceptrons designed to require minimal preprocessing.
Explain the different Layers of CNN.
There are four layered concepts we should understand in Convolutional Neural Networks:
- Convolution: The convolution layer comprises of a set of independent filters. All these filters are initialized randomly and become our parameters which will be learned by the network subsequently.
- ReLu: This layer is used with the convolutional layer.
- Pooling: Its function is to progressively reduce the spatial size of the representation to reduce the number of parameters and computation in the network. Pooling layer operates on each feature map independently.
- Full Connectedness: Neurons in a fully connected layer have full connections to all activations in the previous layer, as seen in regular Neural Networks. Their activations can hence be computed with a matrix multiplication followed by a bias offset.
What is Computational Graph?
A computational graph is a series of TensorFlow operations arranged as nodes in the graph. Each node takes zero or more tensors as input and produces a tensor as output.
Basically, one can think of a Computational Graph as an alternative way of conceptualizing mathematical calculations that takes place in a TensorFlow program. The operations assigned to different nodes of a Computational Graph can be performed in parallel, thus, providing better performance in terms of computations.
What is the significance of a Cost/Loss function?
A cost function is a measure of the accuracy of the neural network with respect to a given training sample and expected output. It provides the performance of a neural network as a whole. In deep learning, the goal is to minimize the cost function. For that, we use the concept of gradient descent.
What Is Data Normalization And Why Do We Need It?
Data normalization is very important preprocessing step, used to rescale values to fit in a specific range to assure better convergence during backpropagation. In general, it boils down to subtracting the mean of each data point and dividing by its standard deviation.
Is Deep Learning is Better than Machine Learning? If so, why?
Though traditional ML algorithms solve a lot of our cases, they are not useful while working with high dimensional data, that is where we have a large number of inputs and outputs. For example, in the case of handwriting recognition, we have a large amount of input where we will have a different type of inputs associated with different type of handwriting.
The second major challenge is to tell the computer what are the features it should look for that will play an important role in predicting the outcome as well as to achieve better accuracy while doing so.
Name a few deep learning frameworks
- TensorFlow
- Caffe
- The Microsoft Cognitive Toolkit/CNTK
- Torch/PyTorch
- MXNet
- Chainer
- Keras
What are some limitations of deep learning?
- Deep learning usually requires large amounts of training data.
- Deep neural networks are easily fooled.
- Successes of deep learning are purely empirical, deep learning algorithms have been criticized as uninterpretable “black-boxes”.
- Deep learning thus far has not been well integrated with prior knowledge.
Which is Better Deep Networks or Shallow ones? and Why?
Both the Networks, be it shallow or Deep are capable of approximating any function. But what matters is how precise that network is in terms of getting the results.
- A shallow network works with only a few features, as it can’t extract more.
- But a deep network goes deep by computing efficiently and working on more features/parameters.
What is Dropout?
Dropout is a regularization technique to avoid overfitting thus increasing the generalizing power. Generally, we should use a small dropout value of 20%-50% of neurons with 20% providing a good starting point. A probability too low has minimal effect and a value too high results in under-learning by the network.
Use a larger network. You are likely to get better performance when dropout is used on a larger network, giving the model more of an opportunity to learn independent representations.
What is Exploding Gradient Descent?
Exploding gradients are a problem when large error gradients accumulate and result in very large updates to neural network model weights during training.
Gradient Descent process works best when these updates are small and controlled. When the magnitudes of the gradients accumulate, an unstable network is likely to occur, which can cause poor prediction of results or even a model that reports nothing useful what so ever.
What’s the difference between a feed-forward and a backpropagation neural network?
A Feed-Forward Neural Network is a type of Neural Network architecture where the connections are “fed forward”, i.e. do not form cycles. The term “Feed-Forward” is also used when you input something at the input layer and it travels from input to hidden and from hidden to the output layer.
Backpropagation is a training algorithm consisting of 2 steps:
- Feed-Forward the values.
- Calculate the error and propagate it back to the earlier layers.
So to be precise, forward-propagation is part of the backpropagation algorithm but comes before back-propagating.
What is gradient descent?
Gradient descent is an optimization algorithm used to minimize some function by iteratively moving in the direction of steepest descent as defined by the negative of the gradient.
- Stochastic Gradient Descent: Uses only a single training example to calculate the gradient and update parameters.
- Batch Gradient Descent: Calculate the gradients for the whole dataset and perform just one update at each iteration.
- Mini-batch Gradient Descent: Mini-batch gradient is a variation of stochastic gradient descent where instead of single training example, mini-batch of samples is used. It’s one of the most popular optimization algorithms.
