Lecture 13 - Explainable ML

Question 1

What is interpretability?

Answer 1

Ability to explain or to present in understandable terms to a human:

• The degree to which a human can understand the cause of a decision
• The degree to which a human can consistently predict the result of a model

Question 2

What is an explanation?

Answer 2

Answer to a WHY question.

• An explanation usually relates the feature values of an instance to its model prediction in a
  humanly understandable way.

Question 3

Why do we need XAI?

Answer 3

Scientific Understanding,
Bias / fairness issues,
Model debugging and auditing,
Human-AI cooperation / acceptance,
Regulatory compliance,
High-risk applications & regulated industries.

Question 4

What is Intrinsic Interpretability?

Answer 4

A model is inherently easy to understand due to its simplicity, without requiring additional tools.

Question 5

What is Post-hoc Interpretability?

Answer 5

Another method or model is used after training to interpret the predictions of a complex or opaque model.

Question 6

What is Model-Agnostic?

Answer 6

Interpretation methods that can be applied to any model, treating it as a black-box.

Question 7

What is Model-Specific?

Answer 7

Interpretation methods that require access to and make use of a model’s internal structure or components.

Question 8

What does Local (Instance-Level) explains?

Answer 8

Explains individual predictions.

Question 9

What does Global (Model-Level) explains?

Answer 9

Explains the overall model behavior.

Question 10

What does Intermediate (Group-Level) explains?

Answer 10

Explains predictions for groups or subsets of data.

Question 11

What does “Result of the interpretation method,” refer to?

Answer 11

refers to the form of explanation produced by XAI techniques.

Question 12

Name five result of the interpretation method

Answer 12

1. Result of the interpretation method
2. Feature summary
   visualization
3. Model internals
4. Data points
5. Global or local surrogates via intrinsically interpretable models

Question 13

What is the result of interpretation methods that provide a feature summary statistic?

Answer 13

• Feature importance (E[feature importance]).
• Feature interaction strengths.
  These summarize the influence or relationships between features across the model.

Question 14

What does a feature summary visualization show?

Answer 14

• Partial dependence plot (PDP): Shows the marginal effect of a feature on the predicted outcome.
• Feature importance plot: Ranks features based on their influence on the model’s predictions.

Question 15

What is provided by interpretation methods that show model internals?

Answer 15

• Linear model weights: Reflect the importance of features in linear models.
• Decision tree structure: Provides “if-then” rules and decision paths.
• Filters: Commonly found in neural networks, where filters capture feature patterns (e.g., in convolutional neural networks).

Question 16

What is the result of interpretation methods focused on data points?

Answer 16

• Exemplars: Representative examples of data points that explain model behavior.
• Counterfactual explanations: Show how changing certain features would change the model’s prediction.

Question 17

What is the purpose of global or local surrogates?

Answer 17

Surrogates are interpretable models (like decision trees or linear models) that approximate the behavior of a complex “black-box” model, providing explanations at a global (whole model) or local (individual instance) level.

Question 18

What does Expressive Power refer to in explanation methods?

Answer 18

Expressive Power refers to the “language” or structure of the explanations provided.

Question 19

What does Translucency describe in the context of explanation methods?

Answer 19

Translucency describes how much the explanation method relies on looking into the machine learning model itself.

Question 20

What does Portability describe in explanation methods?

Answer 20

Portability refers to the range of machine learning models that an explanation method can be used with.

Question 21

What does Algorithmic Complexity measure in explanation methods?

Answer 21

Algorithmic Complexity refers to the computational complexity of the explanation method, indicating how much time and resources it takes to compute the explanations.

Question 22

What does Accuracy refer to in individual explanations?

Answer 22

Accuracy refers to how well an explanation can predict unseen data.

It measures whether the explanation can generalize beyond the examples used to generate it.

Question 23

What is Fidelity in the context of individual explanations?

Answer 23

Fidelity measures how well the explanation approximates the predictions of the black-box model.

High fidelity means the explanation closely mirrors the behavior of the original model

Question 24

What does Certainty/Confidence indicate in an explanation?

Answer 24

Certainty or confidence reflects whether the explanation captures the certainty of the machine learning model.
it should indicate how confident the model is in its prediction, particularly in uncertain or borderline cases.

Question 25

What does Comprehensibility/Plausibility refer to in explanations?

Answer 25

Comprehensibility or plausibility refers to how well humans can understand the explanation and how convincing or trust-building it is.

Question 26

What are some ways to measure Comprehensibility/Plausibility?

Answer 26

1. User studies or surveys: Ask humans to evaluate how understandable or believable they find the explanations. 2. task performance: Measure how well humans can make decisions or predictions based on the explanations. 3. trust metrics: Analyze how much confidence users gain in the system after seeing explanations.

Question 27

How is Fidelity measured in explainability?

Answer 27

Fidelity measures how closely an explanation matches the predictions of the black-box model. - it should be measured objectively by comparing the explanation's output with the actual model's predictions.

Question 28

How is Plausibility is measured in explainability?

Answer 28

Through user studies where participants evaluate whether they find the explanation credible.

Question 29

How is Comprehensibility evaluated in explainability?

Answer 29

It requires a user study to assess whether people can interpret and make sense of the explanation.

Question 30

What is simulatability, and how is it measured?

Answer 30

Simulatability measures the degree to which a human can predict or calculate the model's outcome given the explanation. It is assessed by testing whether humans can reproduce the mode's predoctions based solely on the explanation provided, without needing access to the model itself.

Question 31

What does it mean for an explanation to be contrastive?

Answer 31

A contrastive explanation requires a point of reference for comparison. It answers the question "why this and not that"

Question 32

What does it mean for an explanation to be selective?

Answer 32

A selective explanation is precise, focusing on a small set of the most important factors. Humans can process only around 7±2 cognitive entities at a time, so good explanations are concise.

Question 33

Why is social context important for a good explanation?

Answer 33

A good explanation considers the social context of audience. It tailors the explanation to the understanding, values, or expectations of the explainee.

Question 34

What does it mean for an explanation to be truthful?

Answer 34

A truthful explanation is scientifically sound and proves to be true in reality. Good explanations must accurately reflect the underlying phenomena or model behavior.

Question 35

What makes an explanation General and Probable?

Answer 35

A good explanation should be general, meaning it can explain many events and has a high likelihood of being valid across different situations. General causes are often considered good explanations because they apply broadly.

Question 36

Why should a good explanation be consistent with prior beliefs?

Answer 36

A good explanation is consistent with the prior beliefs if the explainee, making it easier for them to accept and understand. It aligns with what the audience already knows or expects, making it more plausible and convincing.

Question 37

What are interpretable models?

Answer 37

Models that are transparent and simple enough to understand. They stand for their own explanation and reflect perfect fidelity since the model is its own explanation.

Question 38

What are black-box models, and why do they require post-hoc explanations?

Answer 38

Black-box models are complex and opaque, making their internal processes difficult to understand. They require post-hoc explanations because the model itself is not interpretable. These explanations often cannot achieve perfect fidelity with the original model, meaning they may not fully reflect the model's decision-making process.

Question 39

Why might the explanations for black-box models not make sense?

Answer 39

Post-hoc explanations may be oversimplified or inconsistent with the model's actual processes.

Question 40

Why are black-box models often not suitable for combining external information with a risk assessment?

Answer 40

This is because they lack transparency and cannot easily incorporate or explain decisions involving external data.

Question 41

What are intrinsically interpretable models?

Answer 41

Intrinsically interpretable models are models that are transparent and easy to understand without needing post-hoc explanations. Their structure and decision-making process can be understood by humans at a glance.

Question 42

What is a basic requirement for interpretability regarding features/input?

Answer 42

The features or input used by the model must be intelligible and humanly understandable. This means the input data should be in a form that humans can easily interpret.

Question 43

What type of model structure is required for interpretability?

Answer 43

The model must be transparent and simple to understand.

Question 44

Why is a compact set of predictive feature important for interpretability?

Answer 44

A compact set of predictive features means the model uses a small number of key features to make prediction. Fewer, more important features make the model easier to interpret and understand.

Question 45

What is a Linear Model?

Answer 45

A linear model predicts the target variable as a weighted sum of the input features. The relationship between the features and the target is linear.

Question 46

What are Scoring Systems in the hierarchy of linear models?

Answer 46

Scoring systems are linear models where the weights are constrained to be integers.

Question 47

What is a Generalized Additive Model (GAM)?

Answer 47

A GAM is a model where the relationship between the input features and the target is additive but nonlinear. It combines univariate component functions 𝑓𝑖 that can be nonlinear, but the effects are added together

Question 48

What is a link function in Generalized Additive Models?

Answer 48

A link function ℎ(.) connects the linear predictor (sum of the transformed features) to the expected value of the outcome variable. It allows for the target variable to have a non-normal distribution.

Question 49

How do GAMs differ from linear models?

Answer 49

While linear models assume a strict linear relationship between features and the target, GAMs allow for nonlinear relationships through univariate component functions that transform individual features.

Question 50

What is the role of the component functions 𝑓𝑖 in a GAM?

Answer 50

component functions 𝑓𝑖 in a GAM are univariate functions that model the relationship between each feature 𝑥𝑖 and the target. These functions can capture nonlinear patterns in the data for each individual feature.

Question 51

What is the key difference between Linear Models and Generalized Additive Models (GAMs)?

Answer 51

The key difference is that Linear Models assume a linear relationship between features and the target, while GAMs allow for nonlinear relationships by applying smooth functions to the features.

Question 52

What are Generalized Linear Models (GLMs)?

Answer 52

GLMs extend linear models by allowing for non-normal distributions of the response variable. They use a link function to relate the linear predictor to the expected value of the outcome.

Question 53

How do Generalized Linear Models (GLMs) handle nonlinear relationships?

Answer 53

GLMs handle nonlinear relationships by creating new higher-order variables (e.g., polynomial terms) from the original features and applying a linear model in this new feature space.

Question 54

How is the higher-dimensional transformation in GLMs similar to kernel methods in SVM?

Answer 54

Both GLMs and kernel methods involve transforming the feature space to better capture complex relationships.

Question 55

What is the purpose of creating higher-order variables in GLMs?

Answer 55

Higher-order variables in GLMs allow the model to capture nonlinear relationships between features and the target, while still using a linear predictor in the transformed feature space.

Question 56

slide 7/29

Answer 56

nog maken