5.6. Feature Scaling

🧠 Theory: If one feature ranges from 1–1000 and another from 0–1, the model will give more importance to the larger numbers. Feature scaling fixes this.

Feature scaling is the method used to ensure that each feature contributes equally to the analysis in a machine learning model. When features have different ranges, they can skew the results of the model, causing it to favor features with larger numeric ranges. For example, if one feature represents income (range: 1 to 1000) and another represents age (range: 0 to 1), the model might prioritize income due to its broader numerical range, potentially leading to misleading conclusions.

Think of a race between two runners, one running in a flat field and the other uphill. If we only consider the distance each runner covers without taking into account the difficulty of their terrain, it will misrepresent their true abilities. Feature scaling adjusts for these differences, similar to leveling the playing field for both runners.

Two main techniques: ● Normalization: Scale values between 0 and 1

Normalization is a scaling technique that transforms features to fit within a specified range, usually between 0 and 1. This is particularly useful when your data follows different distribution or ranges. Normalization is accomplished using a formula that adjusts each value in relation to the minimum and maximum values in the feature. When all features are on the same scale, the learning algorithm performs more effectively.

Imagine you have a recipe that requires measuring ingredients in cups for baking. If one ingredient is measured in teaspoons (a smaller unit) and another in cups (a larger unit), it becomes challenging to understand the total mix. Normalization helps convert all quantities to the same measuring unit (0 to 1 scale), making it easier to prepare an even mixture.

● Standardization: Mean = 0, Standard Deviation = 1

Standardization transforms features to have a mean of 0 and a standard deviation of 1. This is done by subtracting the mean of the feature from each individual value and then dividing by the standard deviation. Standardization is particularly beneficial for algorithms that assume a normally distributed dataset. This technique can help improve model performance by centering the data, thereby making it easier for models to learn from it.

Consider adjusting the height of students in a classroom to ensure that everyone's height is in a similar range for a group activity. You measure each height, find the average and adjust each height to reflect how far it is from the average. This way, you ensure that no individual's height dominates the activity, allowing for better coordination and collaboration.

✅ Code Example (Standardization): from sklearn.preprocessing import StandardScaler scaler = StandardScaler() X_train_scaled = scaler.fit_transform(X_train) X_test_scaled = scaler.transform(X_test) print(X_train_scaled)

In this code example, we use the StandardScaler from the sklearn library to standardize our training and test datasets. The fit_transform() method calculates the mean and standard deviation of the training data and scales it accordingly. The transform() method applies the same scaling parameters to the test data, ensuring that both datasets are on the same scale.

Imagine a teacher adjusting the test scores of the students against a curve. If the average score is set to a passing grade of 70 and all students’ scores are adjusted accordingly, everyone’s performance can be evaluated fairly. The StandardScaler works similarly by ensuring that scores (or features) are compared on the same scale, enabling the machine learning model to evaluate them equitably.

--