4.2.5. Model Evaluation and Testing

Try these first

1. 1.

   What does a confusion matrix display?

   Hint

   Think about the terms related to classification accuracy.

2. 2.

   Why is cross-validation important when training a model?

   Hint

   Consider what might happen if we only trained on one dataset.

3. 3.

   What does a confusion matrix help to assess?

   • Overall accuracy
   • True and false classifications
   • Data preprocessing needs

   Hint

   Consider what it breaks down in terms of actual and predicted values.

4. 4.

   True or False: Cross-validation improves the accuracy of the final model.

   • True
   • False

   Hint

   Think about how many times the model trains.

5. 5.

   Create a detailed confusion matrix for a model that predicts whether a patient has a disease based on symptoms, providing a hypothetical dataset's true and predicted values. Calculate and interpret accuracy, precision, and recall.

   Hint

   Remember to establish the numbers based on logical assumptions concerning the dataset.

6. 6.

   Design an experiment using cross-validation on a dataset of your choice, outlining how you would structure the dataset and the number of folds. Discuss potential biases and their impact.

   Hint

   Consider how overfitting could mislead the outcomes if not addressed with proper folds.