16.1. Covariance

Covariance is a measure of the joint variability of two random variables. If greater values of one variable correspond to greater values of the other variable (and vice versa), the covariance is positive. If one increases while the other decreases, the covariance is negative.

Covariance helps us understand how two variables change together. If both variables tend to increase together or decrease together, the covariance will be positive. This means there is a direct relationship between the two variables. On the other hand, if one variable tends to increase while the other decreases, the covariance is negative, signifying an inverse relationship.

Think of the relationship between temperature and ice cream sales. When the temperature rises (one variable increases), ice cream sales also tend to rise (the second variable increases). This would give a positive covariance. Conversely, if we look at temperature and hot chocolate sales, when the temperature rises, hot chocolate sales usually drop. This inverse relationship would result in a negative covariance.

Let 𝑋 and 𝑌 be two random variables with means 𝜇 and 𝜇. The covariance is defined as: Cov(𝑋,𝑌) = 𝐸[(𝑋−𝜇𝑋)(𝑌−𝜇𝑌)] For a sample of 𝑛 observations: Cov(𝑋,𝑌) = (1/n) * Σ(𝑖=1 to n) [(𝑥𝑖−𝑥̄)(𝑦𝑖−𝑦̄)] Where: • 𝑥̄ is the mean of 𝑥-values • 𝑦̄ is the mean of 𝑦-values • 𝑥𝑖,𝑦𝑖 are the ith data points.

The formula for covariance involves taking the average of the product of the deviations of each variable from their respective means. The expected value notation (𝐸) signifies taking an average over the dataset. When we calculate it for a sample, we sum the products of each pair of deviations and divide by the number of observations. This quantifies how each variable's movement relates to the other's movement around their averages.

Imagine tracking the heights of students and their reading scores. To compute the covariance, you measure how far each student's height is from the average height and how far their reading score is from the average reading score. If students shorter than average usually scored lower than average, this would indicate a negative covariance, as the height and scores covary in opposite directions.

Covariance Value Interpretation > 0 Positive relationship (direct) < 0 Negative relationship (inverse) = 0 No linear relationship ⚠️ Limitation: Covariance tells us the direction of the relationship but not its strength or consistency.

Covariance values can be interpreted to understand the nature of the relationship between variables. A positive covariance indicates a direct relationship, meaning they move together. A negative covariance shows an inverse relationship, while a value of zero indicates no linear relationship. However, a key limitation is that while we can determine the direction, covariance does not provide clarity on how strong this relationship is.

Consider two friends who often go to the movies together. If they frequently attend movies around the same time, they would have positive covariance in their outings. If one friend goes alone when the other cannot, negative covariance might indicate their patterns are inversely related. However, just knowing they go together or apart does not tell us how close or frequent their outings are.

--