7. Supervised Learning – Logistic Regression

Logistic Regression is a supervised machine learning algorithm used for binary classification problems. It is used when the output variable is categorical, like: ● Yes or No ● Pass or Fail ● 0 or 1 ● Spam or Not Spam Despite its name, logistic regression is not used for regression problems. It is a classification technique.

Logistic Regression is a method used in machine learning to classify data into two distinct categories. For instance, it helps in determining whether an email is spam or not, or whether a student has passed or failed a test. Instead of predicting continuous outcomes, like someone’s salary, it predicts whether something belongs to a certain class or category. This is why it’s categorized as a classification technique rather than a regression technique, despite having 'regression' in its name.

Imagine a teacher who can predict if students will pass or fail based on their study hours. Instead of giving a specific grade, the teacher only wants to know if they will pass (Yes) or fail (No). This scenario perfectly fits logistic regression, as the output is categorical.

Feature | Regression | Classification Output | Continuous values (e.g., salary) | Categories (e.g., pass/fail) Example Algorithm | Linear Regression | Logistic Regression

In machine learning, 'regression' and 'classification' are two different approaches for analyzing data. Regression is used when we want to predict a continuous value, such as predicting someone’s salary based on their years of experience. On the other hand, classification is used when we need to categorize data into discrete classes, like determining if an email is spam or not. Logistic regression, specifically, falls under classification as it predicts which class an observation belongs to.

Think of a school deciding whether a student graduates based on their grades. If the result is a specific percentage (like 75% or higher), it's a regression problem (continuous outcome). If the outcome is just 'Graduate' or 'Not Graduate', it's a classification problem.

Logistic regression uses the sigmoid function to map predicted values to probabilities. σ(z) = 1 / (1 + e^(-z)) Where: ● z = w1x1 + w2x2 + ⋯ + wn*xn + b ● σ(z) ∈ (0,1) — probability of belonging to class 1 If output > 0.5, classify as 1 (Positive) If output < 0.5, classify as 0 (Negative)

The sigmoid function is crucial in logistic regression as it takes any input value (like the weighted sum of features) and maps it to a range between 0 and 1. This mapping to a probability is what allows us to classify outputs. If the calculated probability is greater than 0.5, it means the data point is more likely to be in the 'Positive' class (class 1). Conversely, if it's less than 0.5, it's classified as 'Negative' (class 0).

Imagine you have a magical scale from 0 to 1 that tells you how likely it is to rain tomorrow. If it shows above 0.5, you take an umbrella (it’s likely to rain); if it shows below 0.5, you don’t. That’s similar to how the sigmoid function works in logistic regression.

Step 1: Import Libraries import pandas as pd import numpy as np import matplotlib.pyplot as plt from sklearn.linear_model import LogisticRegression from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score, confusion_matrix Step 2: Create the Dataset data = { 'Hours_Studied': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 'Passed': [0, 0, 0, 0, 0, 1, 1, 1, 1, 1] } df = pd.DataFrame(data)

To build a logistic regression model, we first need to import necessary libraries. Libraries like Pandas are used for data manipulation, Numpy for numerical calculations, and Matplotlib for visualization. After importing these libraries, we create a dataset that contains hours studied by students and whether they passed (1) or not (0). This dataset then serves as the foundation on which we train our model.

Creating a dataset is similar to preparing ingredients for a recipe. Just as you need the right ingredients to cook a delicious meal, you need the right data to develop an excellent machine learning model.

plt.scatter(df['Hours_Studied'], df['Passed'], color='blue') plt.xlabel("Hours Studied") plt.ylabel("Passed (1 = Yes, 0 = No)") plt.title("Hours Studied vs Passed") plt.grid(True) plt.show() You will see a clear pattern — after a certain number of study hours, students are more likely to pass.

Visualizing the data helps us understand patterns and relationships between variables. In this case, plotting the hours studied against whether students passed reveals that as study hours increase, the likelihood of passing also increases. This visual representation helps in formulating hypotheses about the data.

If you're trying to find the relationship between exercise and weight loss, plotting this data in a graph can quickly show you that more exercise often leads to more weight loss. Similarly, a clear pattern in our study data shows how study hours can impact passing rates.

Prepare Features and Labels: X = df[['Hours_Studied']] # Independent variable y = df['Passed'] # Target variable X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) Train the Model: model = LogisticRegression() model.fit(X_train, y_train)

In this step, we prepare our data for model training. The independent variable (input feature) is 'Hours_Studied', and the target variable (output we want to predict) is 'Passed'. We split our dataset into training and testing sets, which will be used to train the model and evaluate its performance, respectively. Finally, we create a logistic regression model and fit it to our training data.

It's like a student studying from a textbook for an exam. They first learn (train) using one portion of the book (training set) and then take a practice test on a different part (test set) to see how well they’ve learned the material.

y_pred = model.predict(X_test) print("Predictions:", y_pred) print("Actual: ", list(y_test)) You can also predict for a new student: print("Will a student who studies 4.5 hours pass?") print("Prediction:", model.predict([[4.5]]))

After training our model, we can now make predictions on new data. Using the test set, we predict outcomes based on the hours studied. This allows us to compare predicted outcomes with the actual results. Additionally, we can ask the model about new cases, such as predicting if a student who studies for 4.5 hours will pass.

Think of this process like a fortune teller predicting the future based on past trends. If they have seen that studying a certain amount leads to passing, they can take that information to make a prediction for someone new coming to them for advice.

Accuracy Score: print("Accuracy:", accuracy_score(y_test, y_pred)) Confusion Matrix: cm = confusion_matrix(y_test, y_pred) print("Confusion Matrix:\n", cm) A confusion matrix shows: ● True Positives (TP) ● True Negatives (TN) ● False Positives (FP) ● False Negatives (FN)

After making predictions, we evaluate how well our model performed. The accuracy score tells us the proportion of correct predictions. Additionally, the confusion matrix provides a detailed breakdown of the model's performance, categorizing the predictions into true positives, true negatives, false positives, and false negatives. This helps us understand where the model might be making mistakes.

Imagine you’re grading an exam. The accuracy score tells you how many students passed based on their scores. The confusion matrix helps you analyze which students struggled with particular questions, revealing insights into potential areas of improvement.

x_values = np.linspace(0, 11, 100).reshape(-1, 1) y_probs = model.predict_proba(x_values)[:, 1] plt.plot(x_values, y_probs, color='red') plt.scatter(df['Hours_Studied'], df['Passed'], color='blue') plt.xlabel("Hours Studied") plt.ylabel("Probability of Passing") plt.title("Logistic Regression Curve") plt.grid(True) plt.show()

To visualize how well the logistic regression model works, we plot the logistic curve. This shows the predicted probabilities of passing based on the hours studied. The curve demonstrates how likely it is for students to pass as study hours increase, providing an intuitive visual representation of the model's predictions.

Think of this curve like a road leading to a destination (passing the exam). As you travel down the road (study more hours), you have a higher probability of reaching your destination (passing the exam). The gradual slope of the curve shows that as you study more, your chances of success increase.

Concept | Description Logistic Regression | Binary classification algorithm Sigmoid Function | Converts output to probability Threshold Value | (like 0.5) to assign class Accuracy | Overall correct predictions Confusion Matrix | TP, TN, FP, FN table

In summary, logistic regression is a powerful tool for binary classification problems. It uses the sigmoid function to convert a linear regression output into a probability, allowing for clear class assignments based on the defined threshold value. The effectiveness of a logistic regression model is often measured by its accuracy and the details provided by a confusion matrix.

Think of logistic regression like a reliable on/off switch for lighting in your room. The sigmoid function adjusts the forecast of how 'on' or 'off' the light should be, and the accuracy score tells you how often the light correctly turns on when it should (or off when it shouldn't). The confusion matrix shows you where things might have gone wrong.

--