Logistic Regression

Question 1

What type of machine learning algorithm is logistic regression?

Answer 1

It is a supervised machine learning algorithm used for classification.

Question 2

What is the primary output of a logistic regression model?

Answer 2

It produces probabilities over one or multiple classes, which are used to categorize data points.

Question 3

For what common task is logistic regression often used as a baseline model?

Answer 3

It is often used to create a baseline for classification tasks and to interpret the effect of input variables on the output.

Question 4

What kind of input data does logistic regression require?

Answer 4

A dataset where features are continuous or categorical, but must be represented numerically.

Question 5

Identify two common use cases for logistic regression.

Answer 5

Credit scoring, medical diagnosis, and spam detection are common use cases.

Question 6

What are the three core concepts a junior data scientist should master for logistic regression?

Answer 6

Loss function, gradient descent, and basic evaluation methods like accuracy, precision, and recall.

Question 7

What is the first step in the process of training a logistic regression model?

Answer 7

Initialize the weights (according to the number of input features) and a bias term to random numerical values.

Question 8

Which function is minimized during the training of a logistic regression model?

Answer 8

The cross-entropy loss function is minimized on the training set.

Question 9

The process of minimizing the loss function in logistic regression involves multiple iterations of _____.

Answer 9

gradient descent

Question 10

What are the gradients of the cross-entropy loss with respect to the weights and bias used for?

Answer 10

They are used to perform the gradient descent update step.

Question 11

Name one common stopping criterion for the training process of a logistic regression model.

Answer 11

A fixed number of epochs, convergence of the loss function, or maximum allowed training time.

Question 12

What activation function is typically used in logistic regression for binary classification?

Answer 12

The sigmoid function.

Question 13

What activation function is typically used in logistic regression for multiclass classification?

Answer 13

The softmax function.

Question 14

How do linear regression and logistic regression differ in their loss functions?

Answer 14

Linear regression minimizes Mean Squared Error (MSE), while logistic regression minimizes Cross-Entropy Loss (log loss).

Question 15

What is the purpose of the vectorized equation $z = w^T x + b$ in logistic regression?

Answer 15

It computes the linear combination of input features and weights, plus a bias term, before applying the activation function.

Question 16

What does the cross-entropy loss function measure?

Answer 16

It measures the difference between predicted probabilities and actual labels across all training examples.

Question 17

In the binary cross-entropy loss graph, what happens to the loss as the predicted probability approaches 0 when the true label is 1?

Answer 17

The loss approaches infinity.

Question 18

In the binary cross-entropy loss graph, what happens to the loss as the predicted probability approaches 0 when the true label is 0?

Answer 18

The loss decreases, approaching 0.

Question 19

Why is the convexity of the binary cross-entropy loss function advantageous for logistic regression?

Answer 19

It ensures that gradient descent can find a single global minimum.

Question 20

Minimizing the binary cross-entropy loss is theoretically equivalent to maximizing what?

Answer 20

It is equivalent to maximizing the likelihood of the observed data.

Question 21

What do the equations for $dL/dW$ and $dL/db$ calculate in logistic regression?

Answer 21

They calculate the gradients of the loss with respect to the weights and bias, used for the backward pass.

Question 22

What is the formula for the gradient descent update step for the weights in logistic regression?

Answer 22

$w_n = W - \alpha \cdot \frac{dL}{dW}$

Question 23

What is the purpose of the sigmoid function in binary logistic regression?

Answer 23

It transforms the unbounded linear output (logits) into a probability between 0 and 1.

Question 24

What is the output of the sigmoid function for an input of 0?

Answer 24

The output is 0.5, representing maximum uncertainty.

Question 25

What is the formula for the sigmoid activation function?

Answer 25

$\sigma(z) = \frac{1}{1 + e^{-z}}$

Question 26

The softmax function is used for _____ classification, while the sigmoid function is used for _____ classification.

Answer 26

multiclass, binary

Question 27

What is a key property of the probabilities output by the softmax function?

Answer 27

The sum of all the output probabilities is equal to 1.

Question 28

What is a major limitation of logistic regression that can be mitigated by L1/L2 regularization?

Answer 28

Overfitting.

Question 29

How does regularization help mitigate overfitting in logistic regression?

Answer 29

It adds a weight penalty to the loss term, encouraging smaller weights and thus reducing variance.

Question 30

What key assumption does logistic regression make about the relationship between inputs and the log odds of the prediction?

Answer 30

It assumes a linear relationship.

Question 31

Name an alternative algorithm to use when the assumptions of logistic regression, like linearity, are not met.

Answer 31

Decision trees or ensemble algorithms like Random Forest.

Question 32

How can sensitivity to imbalanced training data be addressed in logistic regression?

Answer 32

By undersampling the majority class, oversampling the minority class, or applying class weights during training.

Question 33

In the `scikit-learn` library, what class is used to implement a logistic regression model?

Answer 33

The `LogisticRegression` class from `sklearn.linear_model`.

Question 34

In `scikit-learn`, which method is called on a `LogisticRegression` object to train it with data?

Answer 34

The `.fit(X_train, y_train)` method.

Question 35

How are the coefficients in logistic regression interpreted?

Answer 35

They indicate the change in the log-odds of the outcome for a one-unit increase in the predictor variable.

Question 36

A _____ coefficient in logistic regression increases the likelihood of the outcome.

Answer 36

positive

Question 37

Why is accuracy a poor evaluation metric for a spam email classifier where most emails are not spam?

Answer 37

A model can achieve high accuracy by simply classifying all emails as non-spam, giving a false sense of good performance.

Question 38

What three metrics are better suited than accuracy for evaluating a spam classifier with imbalanced data?

Answer 38

Precision, recall, and the F1 score.

Question 39

What is the relationship between minimizing the cross-entropy loss function and the maximum likelihood principle?

Answer 39

Minimizing the cross-entropy loss is equivalent to maximizing the log-likelihood of the model's probability distribution given the data.

Question 40

What is an advantage of using minibatch stochastic gradient descent (SGD) regarding memory usage?

Answer 40

Smaller batches of a large training set can more easily fit into CPU/GPU memory.

Question 41

What is the effect of using a smaller batch size in minibatch SGD?

Answer 41

Using smaller batch sizes has a regularization effect and can help the model generalize better to unseen data.

Question 42

What is the effect of using a larger batch size in minibatch SGD?

Answer 42

Larger batch sizes provide more accurate gradient estimates and may result in faster convergence.

Question 43

What is the name of the function represented by the formula $L = -\frac{1}{m} \sum_{i=1}^{m} (y_i \log(\hat{y}_i) + (1-y_i) \log(1-\hat{y}_i))$?

Answer 43

The Cross-Entropy Loss function (or log loss).

Question 44

What role does the learning rate, denoted by $\alpha$, play in the gradient descent step equation?

Answer 44

It controls the size of the step taken to update the weights and bias in the direction that minimizes the loss function.

Question 45

In the context of logistic regression, what is the 'forward pass'?

Answer 45

It is the process of making predictions by computing the linear combination of inputs and passing it through an activation function.

Question 46

In the context of logistic regression, what is the 'backward pass'?

Answer 46

It is the process of calculating the gradients of the loss function with respect to the model's parameters (weights and bias).

Question 47

What is the purpose of the `predict(X, W, b)` function in the provided Python code example?

Answer 47

It makes predictions by taking input features, weights, and a bias, computing their dot product, and applying the sigmoid function.

Question 48

What does the `binary_cross_entropy(y_pred, y)` function compute in the Python code?

Answer 48

It computes the binary cross-entropy loss between the predicted probabilities and the ground truth labels.

Question 49

The training loop in the provided Python code iterates over the data in batches. This implementation demonstrates _____.

Answer 49

minibatch stochastic gradient descent

Question 50

What does logistic regression assume about multicollinearity among input features?

Answer 50

It assumes the absence of high multicollinearity between the independent variables.