Enrol to start learning
Reading is open to everyone. Enrolling is free, and it is what unlocks the audio lessons, practice tests and progress tracking.
2.2.1. Types of Missingness
Interactive Audio Lesson
Unlock the classroom podcast
The transcript is above and free to read. A free account plays the conversation back.
Create a free account
Today, we'll discuss the first type of missing data: Missing Completely At Random, or MCAR. This occurs when the likelihood of a data point being missing is completely unrelated to any other information. Can anyone give an example of this?
Maybe when someone accidentally skips a question on a survey?
Or when there's a glitch in data collection that causes random rows to be missing?
Exactly! MCAR has no bias on statistical inference, which is why it’s less problematic. Remember the acronym: MCAR—Missing Completely At Random. It helps in retaining data quality! Why is it important to detect whether data is MCAR?
So we know how to handle it correctly?
That's right! Recognizing MCAR allows you to decide whether to delete missing entries without worries. Let’s summarize: MCAR means data is missing randomly, not impacting our results.
Unlock the classroom podcast
The transcript is above and free to read. A free account plays the conversation back.
Create a free account
Next, we move to Missing At Random, or MAR. In this case, the missingness can be explained by other observed variables. Any examples?
Perhaps if older individuals tend to skip questions about technology on a survey about digital habits?
Or if people don't answer because they might feel embarrassed about their income?
Yes! Although this is more complex, it's still manageable. MAR implies that we might still accurately infer the missing data using the information we have. Remember: MAR—Missing At Random. Why is this concept critical in data analysis?
Because we need to use appropriate approaches, like imputation, to reduce bias.
Correct! It leads to better performance in predictive modeling.
Unlock the classroom podcast
The transcript is above and free to read. A free account plays the conversation back.
Create a free accountNow, let’s discuss Missing Not At Random, or MNAR. This is the trickiest type, as the missingness reflects the value of the missing data itself. Can you think of situations where MNAR might apply?
If someone with a high income did not want to disclose their income due to privacy concerns?
Or in health surveys where individuals with severe symptoms avoid answering certain questions?
Exactly! MNAR makes it difficult to analyze data accurately because the missing values are non-randomly distributed. Remember: MNAR—Missing Not At Random. How should we deal with MNAR cases?
We may need to employ more advanced techniques or accept that it's an intrinsic bias in our data.
Right! MNAR must be treated carefully because it can lead to substantial bias in results. Let's recap: MNAR relates the missingness to the unseen data, leading to potential information loss.
Overview
Short Summary
This section discusses the types of missing data in datasets, specifically MCAR, MAR, and MNAR, and their implications for data analysis.
Medium Summary
The section elaborates on the three primary types of missingness: Missing Completely At Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR), detailing their characteristics and the potential biases each can introduce in data analysis. Understanding these types helps data scientists make informed decisions when handling missing data.
Detailed Summary
Types of Missingness
In data handling, understanding the types of missing values is crucial for accurate analysis. This section details three categories of missingness:
- Missing Completely At Random (MCAR): This means that the missing values are entirely random and not related to any observed or unobserved data. For example, a survey participant accidentally skipping a question would be an MCAR scenario. MCAR has no bias on statistical analyses, making it the ideal case.
- Missing At Random (MAR): Here, the propensity for a data point to be missing is related to some of the observed data but not the missing data itself. For instance, individuals from a specific demographic may not respond to a survey question more often. Although MAR introduces some bias, if handled correctly, it can often be mitigated through proper imputation methods.
- Missing Not At Random (MNAR): In this scenario, the missingness is related to the unseen data itself. For example, high-income individuals might skip questions related to income due to personal reasons. MNAR can lead to significant bias if not addressed properly, as the missing data is inherently connected to the data we analyze.
Understanding these types of missingness is vital in determining the appropriate strategies for handling missing data, which can include deletion, imputation, or using predictive models.
Reference YouTube Videos
Audio Book
Unlock the audio lesson
The script is above and free to read. A free account plays it back, in the voice you pick.
Create a free account• MCAR – Missing Completely At Random
Detailed Explanation
MCAR stands for Missing Completely At Random. This means that the missing data has no relationship with either the observed or unobserved data. In other words, the absence of data is entirely random and not related to the specific values of the data. For instance, if a survey respondent skips a question due to accidentally missing it rather than any bias or systematic error, that data is considered MCAR.
Examples & Analogies
Think about a class where a few students are absent on a random day. Their absence does not relate to their performance or any specific situation; it just happens to be a coincidence. Similarly, in data collection, if some responses are missing just by chance without any influence from other factors, that's MCAR.
Unlock the audio lesson
The script is above and free to read. A free account plays it back, in the voice you pick.
Create a free account• MAR – Missing At Random
Detailed Explanation
MAR stands for Missing At Random. This situation arises when the probability of missing data on a variable is related to some of the observed data but not the missing data itself. For example, women might be less likely to report their income in a dataset that includes both male and female participants, but within those women, the missing income data is random and not dependent on actual income levels.
Examples & Analogies
Imagine you are conducting a health survey. If younger participants are less likely to answer questions about their exercise habits compared to older participants, the missing responses from the younger group do not depend on their actual health habits—this is MAR. The characteristic of age influences whether data is missing, but not the health habits themselves.
Unlock the audio lesson
The script is above and free to read. A free account plays it back, in the voice you pick.
Create a free account• MNAR – Missing Not At Random
Detailed Explanation
MNAR stands for Missing Not At Random. This scenario occurs when the missingness is related to the value of the data that is missing. For instance, people with very high incomes may choose not to disclose their income information in a survey, leading to missing data that specifically correlates to the income variable.
Examples & Analogies
Consider a situation where you're trying to gather information on personal savings, but only those with minimal savings choose to respond. In this case, the missing data is definitely influenced by the variable being examined (savings), making it MNAR. This means the nature of the data missing is directly tied to the characteristics of that data.
--
Key Concepts
Examples
Step-by-step examples to apply the section's ideas and test your understanding.
If a respondent skips a survey question purely by accident, it represents MCAR.
If a survey shows that only younger respondents skip questions about technology, it depicts MAR.
If individuals with higher incomes avoid answering income questions, it illustrates MNAR.
Memory Aids
Interactive tools to help you remember key concepts
Stories
Flash Cards
Glossary
MCAR
Missing Completely At Random; data that is missing without any systematic bias.
MAR
Missing At Random; missing data that can be explained by observed data.
MNAR
Missing Not At Random; missing data that relates to the unobserved parameter itself.
Imputation
The process of replacing missing data with substituted values.