Logistic Regression Machine Learning

What You’ll Learn 📚

• Understanding the basics of Logistic Regression
• Key terminology and concepts
• Step-by-step examples from simple to complex
• Common questions and troubleshooting tips
• Practical exercises to reinforce learning

Introduction to Logistic Regression

Logistic Regression is a statistical method for predicting binary classes. The outcome or target variable is binary, meaning it has two possible types: 0/1, yes/no, true/false, etc. It’s a type of regression analysis used for prediction of outcome of a categorical dependent variable based on one or more predictor variables.

Think of Logistic Regression as a way to classify things into two buckets. 🪣

Core Concepts

• Sigmoid Function: A mathematical function that maps any real-valued number into a value between 0 and 1.
• Odds: The ratio of the probability of an event occurring to the probability of it not occurring.
• Logit: The natural log of the odds.

Key Terminology

• Binary Classification: Classifying data into two distinct classes.
• Decision Boundary: A threshold that helps classify the data points.
• Feature: An individual measurable property or characteristic of a phenomenon being observed.

Getting Started with a Simple Example

import numpy as np
from sklearn.linear_model import LogisticRegression

# Sample data: [hours studied, pass/fail]
data = np.array([[1, 0], [2, 0], [3, 0], [4, 1], [5, 1], [6, 1]])
X = data[:, 0].reshape(-1, 1)  # Feature: hours studied
y = data[:, 1]  # Target: pass (1) or fail (0)

# Create and train the model
model = LogisticRegression()
model.fit(X, y)

# Predicting for a new student who studied 4 hours
prediction = model.predict([[4]])
print('Predicted class for 4 hours of study:', prediction)

Progressively Complex Examples

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Sample data
data = {'Age': [22, 25, 47, 52, 46, 56, 55, 60],
        'Salary': [21000, 25000, 47000, 52000, 46000, 56000, 55000, 60000],
        'Churn': [0, 0, 1, 1, 0, 1, 1, 1]}
df = pd.DataFrame(data)
X = df[['Age', 'Salary']]
y = df['Churn']

# Splitting data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)

# Feature scaling
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# Create and train the model
model = LogisticRegression()
model.fit(X_train, y_train)

# Predicting
predictions = model.predict(X_test)
print('Predictions:', predictions)

Example 3: Handwritten Digit Classification

Let’s take it up a notch and classify handwritten digits using the famous MNIST dataset.

from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Load dataset
digits = load_digits()
X = digits.data
y = digits.target

# Binary classification: digit '0' vs not '0'
y = (y == 0).astype(int)

# Splitting data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=0)

# Create and train the model
model = LogisticRegression(max_iter=10000)
model.fit(X_train, y_train)

# Predicting
predictions = model.predict(X_test)
accuracy = accuracy_score(y_test, predictions)
print('Accuracy:', accuracy)

Common Questions and Troubleshooting

1. What is the difference between logistic and linear regression?

   Logistic regression is used for binary classification, whereas linear regression is used for predicting continuous values.

2. Why do we use the sigmoid function?

   The sigmoid function maps predictions to probabilities, making it suitable for binary classification.

3. How do I interpret the coefficients in logistic regression?

   Coefficients represent the change in the log odds of the outcome for a one-unit change in the predictor variable.

Common Pitfall: Forgetting to scale your features can lead to poor model performance. Always check if scaling is necessary!

Practice Exercises

Try these exercises to reinforce your learning:

• Use logistic regression to predict whether a person has diabetes based on a dataset of medical features.
• Experiment with different feature scaling techniques and observe their impact on model performance.

Remember, practice makes perfect! Keep experimenting and don’t hesitate to revisit concepts as needed. You’ve got this! 🚀