What is back-testing in the context of capital PD model performance?
Back-testing compares model results against actual historic default rates.
What does back-testing help identify?
Trends or one-way bias in model predictions.
What actions may be taken if a model severely underpredicts defaults?
Redevelopment, recalibration, or adjustment through an overlay.
At what level should back-testing analysis be conducted?
At the same level of granularity as that of the model.
What is the implication of a model that over-predicts for one segment and under-predicts for another?
It may reduce the risk of inaccurate estimates at a portfolio level.
What does the Gini statistic measure?
The strength of a model’s ability to distinguish between ‘good’ and ‘bad’ risks.
What Gini coefficient range is considered strong for a retail portfolio?
Between 60% and 70%.
What does a Gini coefficient lower than 50% indicate?
The model may not adequately discriminate between the riskiness of loans.
What does a Gini coefficient higher than 70% suggest?
The model may be overfit to historic data.
What is the purpose of the Chi-Square (Hosmer-Lemeshow) test?
To compare actual versus expected results in model fit.
How are observations sorted in the Chi-Square test?
In increasing order of their estimated probability of default.
What does a high p-value in the Chi-Square test indicate?
A model with a good fit.
What is benchmarking in the context of model validation?
Comparing internal estimates against external sources.
What is important to consider when benchmarking different organizations?
The differences in underwriting and credit risk management processes.
What does Population Stability Index (PSI) measure?
How much the population has changed over time.
What PSI value indicates minimal change in the population?
PSI < 0.1.
What does a PSI > 0.2 indicate?
A significant change in the population.
What should be investigated if a high PSI is detected?
The reasons for population changes.
What is the value of out-of-time and out-of-sample testing?
To ensure the model is performing as expected and not overfit.
What should the validation process reveal?
Changes in drivers, trends, and correlations.
Why are strong risk aggregation capabilities vital?
To ensure business grows only as quickly as control infrastructure.
What responsibility do business line leaders have regarding risk models?
To understand the models and ensure their risks are incorporated into the bank-wide risk process.
