6. Assumptions in Linear Regression

1. Linearity – Relationship between X and y is linear.

The assumption of linearity states that there is a straight-line relationship between the independent variable (X) and the dependent variable (y). This means that changes in X should lead to proportional changes in y. If this assumption is violated, the predictions provided by the linear regression model may not be accurate because the model is not capturing the true relationship.

Imagine you are measuring how much time you spend studying (X) against your score on a test (y). A linear relationship suggests that if you double your study time, your score should also double. If studying for five extra hours yields a huge change in score, but studying for ten hours produces a minimal change, then the linearity assumption does not hold.

2. Homoscedasticity – Equal variance of errors.

Homoscedasticity means that the variability of the errors (the differences between the observed values and the predicted values) should remain constant at all levels of the independent variable. This is important because if the spread of errors increases or decreases systematically with changes in X, it can lead to inefficiency in the model estimation and biased predictions.

Think of a scenario where you’re testing how much a car's fuel efficiency (y) changes with different speeds (X). If the difference between actual and predicted values (errors) gets smaller at lower speeds and larger at higher speeds, that would indicate that we have heteroscedasticity. It would be like measuring the height of plants under different conditions and seeing varying ranges of heights instead of the same level of variability across all conditions.

3. No multicollinearity – Independent variables should not be highly correlated.

Multicollinearity occurs when two or more independent variables in a regression model are highly correlated, meaning they contain similar information. This can create instability in the coefficient estimates and make it difficult to determine the individual effect of each variable on the dependent variable. You want to ensure that each independent variable provides unique information.

Imagine you are trying to predict student performance based on study hours and hours spent on social media. If these two variables are highly correlated (say, students who study more tend to spend less time on social media), it can be confusing to attribute to which factor is affecting student performance. It’s like trying to assess the impact of spice and salt on food flavor when they are both present in similar amounts, making it hard to appreciate the distinct influence of each.

4. Normal distribution of errors.

This assumption states that the errors of the model should be normally distributed around the mean of zero. This is essential for hypothesis testing and for making reliable confidence intervals around the predicted values. If the errors are not normally distributed, it can lead to unreliable estimates and inference.

Think of the results of a standardized test taken by a large group of students. If we expect the scores to cluster around an average with fewer students scoring very high or very low, we anticipate a normal distribution of scores. If instead, most students score very high with some very low, the error in our model may not allow us to accurately reflect performance. Understanding this helps ensure that predictions maintain an expected level of accuracy.

--