Pytorch

Question 1

What is the Pytorch “Module” submodule, and how is it used?

Answer 1

• It’s the base class for all NN modules
• automatic registration of all trainable parameters (PyTorch recognizes the components of my model)

Setup:
1. Creates a class that inherits from torch.nn.Module
2. Define the __init__() method (for layers/submodules & setting up the architecture)
3. Define the forward() method (to specify the flow through the architecture)
4. Create an instance of your class and apply it to input

Question 2

Examples for gradient-based Optimizers (in Pytorch)

Answer 2

• Stochastic Gradient Descent (good baseline)
• Adam (adaptive Learning rate)

Question 3

Loss function for regression tasks

Answer 3

MSE (Mean Squared Error)

typically, no output activation function

Question 4

Loss function for classification tasks

Answer 4

Cross Entropy

sigmoid or log-softmax activation function

Question 5

What is Regularization and what are common examples?

Answer 5

Methods to counter overfitting

• Dropout: random features or inputs
• Weight penalty terms: add additional terms to training loss (L1, L2)
• Noise: add random noise to inputs or features

Question 6

What is a Confusion Matrix?

Answer 6

A table summarizing classification results: True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN).

Question 7

Define False Positive (FP).

Answer 7

Samples wrongly classified as positive (false alarm, Type I error).

Question 8

Define False Negative (FN).

Answer 8

Samples wrongly classified as negative (miss, Type II error).

Question 9

Formula for True Positive Rate (TPR) = Recall = Sensitivity.

Answer 9

“How many true cases did I find?”

TPR = TP / (TP + FN)

Question 10

Formula for True Negative Rate (TNR) = Specificity

Answer 10

“How well do I reject negatives?”

TNR = TN / (TN + FP)

Question 11

Formula for False Negative Rate (FNR).

Answer 11

FNR = FN / (FN + TP) = 1 - TPR.

Question 12

Formula for False Positive Rate (FPR).

Answer 12

FPR = FP / (FP + TN) = 1 - TNR.

Question 13

Formula for Accuracy (ACC).

Answer 13

overall correctness (misleading if imbalanced)
ACC = (TP + TN) / (TP + TN + FP + FN).

Question 14

Formula for Balanced Accuracy (BACC).

Answer 14

BACC = (TPR + TNR) / 2;
average of TPR & TNR (good for imbalance)

Question 15

What is ROC and AUC?

Answer 15

ROC curve shows TPR vs FPR across thresholds; AUC is the area under ROC (0.5 random, 1 perfect).

Question 16

Formula for Positive Predictive Value (Precision).

Answer 16

“How precise are my alarms?”
PPV = TP / (TP + FP).

Question 17

Formula for F1-score.

Answer 17

harmonic mean of Precision & Recall (balance them)

F1 = 2 * (Precision * Recall) / (Precision + Recall) = 2TP / (2TP + FP + FN).

Question 18

Formula for Fβ-score.

Answer 18

Fβ = (1 + β²) * (Precision * Recall) / (β² * Precision + Recall).

Question 19

Formula for Mean Absolute Error (MAE).

Answer 19

MAE = (Σ|y - ŷ|) / n.

Question 20

Formula for Mean Squared Error (MSE).

Answer 20

MSE = (Σ(y - ŷ)²) / n.

Question 21

Formula for Root Mean Squared Error (RMSE).

Answer 21

RMSE = sqrt(MSE).

Question 22

What does MSE emphasize?

Answer 22

It exaggerates the presence of outliers.

Question 23

What is a Hold-Out Test Set?

Answer 23

Split dataset into train, validation, and test sets; test set provides independent performance estimate.

Question 24

What is Cross Validation (CV)?

Answer 24

Split data into n folds; train n times leaving one fold out each time; average validation risk to estimate generalization.

Question 25

What is Transfer Learning?

Answer 25

Reusing knowledge from a pre-trained model on a large source task for a smaller target task.

Question 26

Two main approaches to Transfer Learning.

Answer 26

Option 1: Feature extractor (freeze backbone, train new head). Option 2: Fine-tuning (train more layers or entire model).

Question 27

When to prefer fine-tuning over feature extraction?

Answer 27

If domain differs significantly, enough data is available, or Option 1 fails to achieve desired accuracy.

Question 28

Why use Transfer Learning?

Answer 28

Saves compute power, data, and time; pre-trained models (e.g., HuggingFace) are readily available.

Question 29

What does the sigmoid function do?

Answer 29

"logistic function" Maps the output between 0 and 1. Activation after sigmoid can be interpreted as the probability of belonging to the positive class.

Question 30

Why use PyTorch for machine learning (e.g. instead of NumPy)?

Answer 30

- Provides GPU support (NumPy is CPU only) - dynamic computation graphs with autograd - a Pythonic API like NumPy - and a large ecosystem (TorchVision, TorchAudio)

Question 31

What is a Tensor in PyTorch?

Answer 31

The core data structure, similar to NumPy arrays but with GPU and autograd support.

Question 32

Difference between RAM and VRAM in PyTorch?

Answer 32

VRAM is GPU memory; PyTorch allows moving data to VRAM for parallel computation.

Question 33

What is the goal of training a neural network?

Answer 33

To choose weights so the model output ŷ approximates the true target y by minimizing a loss function L(ŷ, y).

Question 34

Why are iterative methods used in training?

Answer 34

Because we usually cannot compute the loss minimum directly.

Question 35

How does gradient-based optimization work?

Answer 35

Compute loss --> Compute gradients w.r.t. weights --> Update weights by subtracting learning rate times gradient (θ ← θ - η∇L).

Question 36

What is torch.utils.data.Dataset used for?

Answer 36

To create custom datasets with a standardized interface. Must implement __len__() and __getitem__(). --> to access and optionally transform data samples (e.g., applying preprocessing or augmentation)

Question 37

What does __getitem__() in a Dataset return?

Answer 37

- A single sample (e.g., image, label, ID), with optional preprocessing and augmentation. - Shape and type of a single sample can be derived from reading the __getitem__() method

Question 38

What is torch.utils.data.DataLoader?

Answer 38

A DataLoader wraps a Dataset and makes it possible to: - Iterate in minibatches (not one item at a time). - Shuffle data each epoch. - Use multiple workers (processes) to load data in parallel. - Apply collate logic to combine samples into a batch tensor.

Question 39

How to customize batching in DataLoader?

Answer 39

Use the collate_fn argument: It combines the samples extracted by the getitem method into a single minibatch.

Question 40

Typical steps to use DataLoader?

Answer 40

1. Derive a Dataset class with **__getitem__** and **__len__**. 2. Create dataset *instance*. 3. Create **Subset** splits. 4. Create **DataLoader** with *batchsize, shuffle, numworkers*. 5. Loop over DataLoader to get *minibatches*.

Question 41

Best practice regarding shuffle for validation/test sets in DataLoader?

Answer 41

Disable shuffling because evaluation does not depend on sample order.

Question 42

How to ensure reproducibility with DataLoader?

Answer 42

1. Store dataset split indices 2. Use torch.utils.data.Subset (= FIX the indices) 3. Set random seeds.

Question 43

Do you need to convert samples to tensors inside Dataset?

Answer 43

No, DataLoader's default collate function can handle NumPy arrays and convert them to tensors.

Question 44

Why include sample ID in Dataset return value?

Answer 44

For debugging and traceability.

Question 45

What is torch.nn.Module?

Answer 45

Base class for all neural networks and layers; registers parameters and submodules automatically.

Question 46

Which methods must a custom nn.Module implement?

Answer 46

__init__() to define layers and forward() to specify how input is transformed to output.

Question 47

Why call the module instance directly instead of forward()?

Answer 47

It runs forward() and triggers any registered hooks automatically.

Question 48

How to move modules or parameters to GPU?

Answer 48

Use to(device=...) to move them to a GPU device.

Question 49

Where to find many predefined modules?

Answer 49

torch.nn library (https://pytorch.org/docs/stable/nn.html).

Question 50

Describe a Fully-Connected Feed-Forward Neural Network (FFNN).

Answer 50

Every neuron is connected to all neurons in the next layer; high complexity; typically requires flattening multidimensional input.

Question 51

Describe a Convolutional Neural Network (CNN).

Answer 51

Applies kernels (filters) across input dimensions to capture spatial or temporal patterns, often followed by pooling and flattening.

Question 52

Difference between 1D, 2D, and nD CNNs?

Answer 52

1D for sequences (e.g., time series), 2D for images, nD possible but rare.

Question 53

What is a Recurrent Neural Network (RNN)?

Answer 53

Processes sequential inputs, reusing the same weights and hidden state; variants include LSTM and Transformers.

Question 54

Key properties of ReLU.

Answer 54

Computationally cheap and avoids vanishing gradients; most common activation.

Question 55

Key properties of SELU.

Answer 55

Self-normalizing, improving stability of training.

Question 56

Key properties of Sigmoid.

Answer 56

Outputs in [0,1]; older, prone to vanishing gradients.

Question 57

What is autograd?

Answer 57

PyTorch's automatic differentiation engine that builds computation graphs and computes gradients using backward().

Question 58

Which method computes gradients of a loss tensor?

Answer 58

loss.backward().

Question 59

Which optimizers are common in PyTorch?

Answer 59

torch.optim.SGD (with optional momentum) and torch.optim.Adam (adaptive learning rate and momentum).

Question 60

Steps in a weight update cycle?

Answer 60

1. optimizer.zero_grad() 2. loss.backward() 3. optimizer.step()

Question 61

Common loss functions for regression.

Answer 61

Mean Squared Error (torch.nn.MSELoss), typically with no output activation.

Question 62

Common loss functions for classification.

Answer 62

Cross-Entropy (torch.nn.CrossEntropyLoss) or BCEWithLogitsLoss for binary tasks (includes sigmoid internally).

Question 63

Why is BCEWithLogitsLoss used instead of sigmoid + BCELoss?

Answer 63

It combines sigmoid and binary cross-entropy for numerical stability.

Question 64

Difference between update and epoch.

Answer 64

Update = one weight update; Epoch = one full pass through the training data.

Question 65

What is early stopping?

Answer 65

Stop training when validation loss does not improve for m updates/epochs and use the model with best validation loss.

Question 66

What are good classification metrics for imbalanced classes?

Answer 66

Balanced Accuracy (BAC) AUC (Ranking ability across thresholds) F1 (Balance precision & recall) Fβ (Tune recall vs precision)

Question 67

Purpose of monitoring tools like TensorBoard.

Answer 67

Visualize loss, weights, gradients; monitor training and validation performance.

Question 68

First step if the model is not learning.

Answer 68

Check if gradients and weights are changing; consider gradient clipping.

Question 69

Recommended workflow before regularizing.

Answer 69

First find a model that can overfit, then make it smaller or add regularization.

Question 70

What does the Softmax function do?

Answer 70

Converts logits to class probabilities in multi-class classification. Often used together with CrossEntropyLoss

Question 71

What's the output range of - Tanh - sigmoid - softmax - ReLu

Answer 71

**Tanh**: (−1, 1) Zero-centered output, helpful for RNNs or when data is mean-centered. Still can saturate. **sigmoid**: (0, 1) Good for probabilities in binary classification output layer. **ReLu**: [0, ∞) Default in most modern nets. Fast, sparse activations. Watch for “dying ReLU”

Question 72

Typical normalization approaches

Answer 72

- Min-Max scaling ([0, 1] or [-1, 1]) - Standardization (mean=0, std=1. --> most ML models) - Robust scaling (median=0, IQR=1)

Question 73

What can be the reasons if training loss increases?

Answer 73

(It means the model predictions are getting worse (larger error)) Reasons: - Learning rate too high: the optimizer overshoots minima. - Bad initialization or exploding gradients. - Bug in loss computation or data pipeline. - Sudden data distribution change (if you reshuffle with different data properties).

Question 74

Image normalization: min-max scaling formula?

Answer 74

(img - img.min()) / (img.max() - img.min()) or (x - min) / (max - min)

Question 75

z-score formula for normalization

Answer 75

Goal: center the data and give it unit variance. (x - mean) / std

Question 76

Types of gradient descent / training procedures

Answer 76

- **Full-batch** (a.k.a. batch gradient descent) Uses all training samples to compute the gradient before making one parameter update per epoch. - **Mini-batch gradient descent** Splits the dataset into smaller batches (e.g. 32, 64). Each batch produces one gradient estimate and update. - **Online / stochastic gradient descent (SGD)** Uses a batch size of 1 sample for each update

Question 77

What are the two Dataset types in the Pytorch Dataset class?

Answer 77

mapping style datasets iterable datasets

Question 78

What's the difference between torch.jit.trace and torch.jit.script?

Answer 78

torch.jit.trace → “replays” the **route** of the example input; it cannot see data-dependent branching! (if statements) But doesn't keep the input! torch.jit.script → keeps Python control flow and conditions alive.

Question 79

What does torch.utils.data.Subset do?

Answer 79

It wraps an existing Dataset and exposes only the samples whose indices you pass in. subset = Subset(dataset, indices)

Question 80

What's the default input layout of images in PyTorch?

Answer 80

N × C × H × W number of images x number of channels x height x width

Question 81

What's the tuple "batch shape" consist of?

Answer 81

(batch size, sample shape)