9.7. Step 6: Evaluate the Model

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score, confusion_matrix

In this part, we are importing essential functions from the Scikit-learn library. These functions will help us compute various metrics that will allow us to evaluate the performance of our machine learning model. The metrics include accuracy, precision, recall, F1 score, and confusion matrix, each providing different insights into how well our model is performing.

Think of these metrics like a teacher's report card. Just as the teacher assesses a student's performance based on grades, attendance, and behavior, we assess our model's performance using these different metrics.

print("Accuracy:", accuracy_score(y_test, y_pred))

Accuracy is one of the most straightforward metrics. It tells us the proportion of correct predictions made by our model compared to the total predictions. This is calculated by dividing the number of correct predictions by the total predictions made. This gives us a quick sense of how often the model is correct.

Imagine a student answers 80 out of 100 questions correctly on a test. The accuracy would be 80%. It tells us how many answers the student got right, similar to how we assess our model's predictions.

print("Precision:", precision_score(y_test, y_pred))

Precision measures the number of true positive predictions divided by the total number of positive predictions made, including false positives. In other words, it tells us how many of the positive predictions were actually correct. High precision indicates that the model is making more accurate positive predictions.

Think of a doctor diagnosing patients with a disease. If they diagnose 10 patients and only 7 have the disease, the precision is 70%. It's important because we want to ensure that when the model predicts a positive outcome, it is likely to be correct.

print("Recall:", recall_score(y_test, y_pred))

Recall, also known as sensitivity, measures the proportion of true positives that were correctly identified by the model. It is calculated by dividing the number of true positives by the sum of true positives and false negatives. This metric is vital for understanding how well our model captures all relevant cases.

If the same doctor diagnosed 7 out of 10 patients who actually had the disease, the recall would be 70%. While precision focuses on the accuracy of positive predictions, recall emphasizes how many of the actual positives were caught.

print("F1 Score:", f1_score(y_test, y_pred))

The F1 Score is the harmonic mean of precision and recall, and it provides a balance between the two. This metric is particularly useful when we want to find an optimal balance between precision and recall and is often more informative than accuracy is, especially in imbalanced datasets.

Using our previous examples, if the doctor is balancing precision and recall, the F1 score could help them decide how to interpret their diagnosis strategy: focusing on avoiding false positives while ensuring they catch as many actual cases as possible.

print("Confusion Matrix:\n", confusion_matrix(y_test, y_pred))

A confusion matrix provides a visual representation of how well our model's predictions performed against the actual results. It breaks down predictions into four categories: true positives, false positives, true negatives, and false negatives. This helps us understand where the model is making mistakes and where it is succeeding.

Think of a confusion matrix like a detailed report card that shows specific strengths and weaknesses. Instead of just overall scores, it tells you how many questions were answered correctly, incorrectly, or not answered at all, providing important insights for improvement.

--