8.3. Accuracy

📘 Definition: Accuracy is the ratio of correctly predicted observations to the total observations.

Accuracy = $\frac{TP + TN}{TP + TN + FP + FN}$

Accuracy measures how often the model makes correct predictions. It is calculated by dividing the number of correct predictions (both true positives and true negatives) by the total number of predictions made. This formula, $\frac{TP + TN}{TP + TN + FP + FN}$, gives us a single number that represents the model's performance across all classes.

Imagine you are a teacher grading a class of 100 students on a test. If 90 students pass (correctly predicted positives) and 5 students fail but were correctly identified (correctly predicted negatives), your accuracy is calculated by adding those together (95) and dividing by the total number of students (100). This would give you an accuracy of 95%, which might sound great, but it's essential to dig deeper.

Python Code: from sklearn.metrics import accuracy_score accuracy = accuracy_score(y_true, y_pred) print("Accuracy:", accuracy)

This Python code snippet shows how to calculate the accuracy of a model in practice using the accuracy_score function from the sklearn.metrics module. Here, y_true represents the actual labels, and y_pred represents the predicted labels from the model. The accuracy_score function processes the two lists and returns the accuracy, which is printed out.

Think of this code as a recipe. Just as you need to gather your ingredients (y_true and y_pred) and follow steps to bake a cake, running this code combines your actual and predicted data to produce a number that tells you how 'sweet' or successful your model is.

⚠ Warning: Accuracy can be misleading in imbalanced datasets. For example, if 95% of students pass and 5% fail, predicting all as "pass" gives 95% accuracy, but fails to detect the ones who actually fail.

This warning highlights a critical point: accuracy alone might not tell the full story, especially in situations where one class significantly outnumbers the other (i.e., imbalanced datasets). If a model predicts every student as passing when 95% pass and only 5% fail, it will achieve high accuracy but is essentially useless as it fails to identify the few students who are failing.

Consider a scenario where a fire alarm is programmed to only go off if there is a massive fire. If it ignores small fires, it might seem 'accurate' in its operation if it only alarms when the large fire is present, but ultimately it is dangerous, as it fails to alert for early warning signs. This highlights the importance of also checking other metrics.

--