Confusion Matrix
What types of predictions is a Confusion Matrix used to measure?
aipractitioner
Applies to discrete answers, like Classification models (not continuous values)
Confusion Matrix
What are the two axes on a confusion matrix?
aipractitioner
What the model said vs. what the real world said
Confusion Matrix
What is the Accuracy of a model?
aipractitioner
The model was right (true positive and true negative)
Confusion Matrix
What’s the same about Precision and Recall?
aipractitioner
The model said ‘yes’
Confusion Matrix
What’s happening if the model says ‘yes’ too much?
aipractitioner
False positives – bad Precision
Confusion Matrix
What is the Precision of a model?
aipractitioner
How well does a model predict
Confusion Matrix
What’s the ratio considered for Precision?
aipractitioner
true positives vs. false positives
Confusion Matrix
What is invariant about Precision?
aipractitioner
The model said ‘yes’, decide if it was right or not.
Confusion Matrix
What can you minimize by paying attention to Precision?
aipractitioner
Minimize the number of false positives
Confusion Matrix
Example of when high Precision is important?
aipractitioner
Saying the defendant is guilty
Confusion Matrix
What is the Recall of a model?
aipractitioner
Does the model say ‘yes’ when it should say ‘yes’?
Confusion Matrix
What’s the ratio considered for Recall?
aipractitioner
True positives vs. false negatives
Confusion Matrix
What is the invariant about Recall?
aipractitioner
The actual value is ‘yes’. What did the model say?
Confusion Matrix
What’s the hint for remembering “Recall”?
aipractitioner
Recall is Real (real answer is ‘yes’, what did the model say?)
Confusion Matrix
Mental picture for Recall?
aipractitioner
“Don’t miss anything” (don’t forget and miss a real true value)
Confusion Matrix
What can you minimize by paying attention to Recall?
aipractitioner
The number of false negatives (times the model should have said ‘yes’, but didn’t)
Confusion Matrix
Example of when high Recall is important?
aipractitioner
Predicting if you have cancer
Confusion Matrix
How can you optimize a model for both Precision and Recall?
aipractitioner
You can’t, try to balance the model based on your needs.
Confusion Matrix
How do you measure if your model is balanced?
aipractitioner
“F1” is a metric that balances Precision and Recall when both are important.
Confusion Matrix
What is Model Accuracy?
aipractitioner
How well the model guesses correctly
Confusion Matrix
Quantitatively, what is Accuracy?
aipractitioner
Amount of true positives and true negatives compared to all answers
Confusion Matrix
What metric is best when false positives are costly?
aipractitioner
Precision
Confusion Matrix
What metric is best when false negatives are costly?
aipractitioner
Recall
Confusion Matrix
Best metric for model that predicts spam email?
aipractitioner
Precision (guessed ‘true’ correctly)
-
Agents3
-
AI, ML, DL27
-
Concepts27
-
Fine Tuning20
-
Fit19
-
Guardrails5
-
Model Evaluation (1)21
-
Model Evaluation (2)30
-
Pricing15
-
Prompt Engineering (1)19
-
Prompt Engineering (2)19
-
Q Business20
-
RAG6
-
Responsible AI15
-
Training32
-
Auto ML21
-
Comprehend23
-
Rekognition12
-
SageMaker (1)17
-
SageMaker (2)22
-
Transcribe11
-
Confusion Matrix27
-
to_add_somewhere_130
-
to_add_somewhere_233
-
to_add_somewhere_328
