30.4.3. Model Evaluation

Try these first

1. 1.

   What is accuracy in the context of model evaluation?

   Hint

   Think about how many times the model is correct overall.

2. 2.

   Define precision.

   Hint

   It's about how trustworthy the positive predictions are.

3. 3.

   What does the accuracy metric represent?

   • The percentage of correct predictions
   • The number of false predictions
   • Only true positives

   Hint

   Recall the formula for calculating accuracy.

4. 4.

   True or False: A higher AUC value indicates that a model performs poorly.

   • True
   • False

   Hint

   Think about what AUC measures.

5. 5.

   Given a confusion matrix:

   - python

   Try it yourself

             Predicted Positive  Predicted Negative
   Actual Positive       70                   30
   Actual Negative       10                   90

   Calculate accuracy, precision, recall, and F1-score.

   Hint

   Use the formulas for these metrics based on the values from the matrix.

6. 6.

   Discuss the limitations of using only accuracy as a performance metric with examples. Why is it important to consider precision, recall, and F1-score?

   Hint

   Think about situations like medical diagnoses where false negatives can be life-threatening.