Simple Linear Regression

Simple Linear Regression models the relationship between a single input feature and a continuous output by fitting a straight line through the data. The goal: find the optimal slope and intercept that minimize the prediction error across all data points. Use the controls below to adjust the model parameters and watch the error respond in real-time.

y = wx + b
w = Slope (Weight)
b = Intercept (Bias)
In machine learning literature, this equation is formally written as the Hypothesis Function:
hθ(x) = θ0 + θ1x
(Where θ0 is the bias/intercept, and θ1 is the weight/slope)
DATA POINTS: [ 20 ]
ACTIVE EQUATION: [ y = 1.0x + 2.0 ]
MEAN SQUARED ERROR (MSE): [ --- ]
J(θ0, θ1) = (1 / 2m) Σ (hθ(x(i)) - y(i))² The goal of our model is to minimize this Cost Function J(θ).
  Slope (w) 1.0
  Intercept (b) 2.0
Loading Python Environment...

[ APPLIED LINEAR REGRESSION ]

Use NumPy or Scikit-Learn to fit a line to the provided X and y arrays and print the slope and intercept.

Incoming Transmission...

To fit a model using Scikit-Learn, you first need to instantiate the LinearRegression class from sklearn.linear_model. Then, call the .fit() method passing in your features and target arrays.

model = LinearRegression()
model.fit(X, y)

After fitting, the parameters you need are stored in model.coef_ (slope) and model.intercept_ (bias). Print these out to complete the objective.

Decryption Complete...

The Scikit-Learn Workflow:

Note: In the real world, engineers don't calculate the math from scratch every time. They use highly optimized libraries. Here is how you summon a pre-built Machine Learning model.

  • Step 1: The Toolboxes (import)
    Concept: We need our math toolbox (numpy) for handling the numbers, and our machine learning toolbox (sklearn) to give us the actual algorithm.
    Action: 
    import numpy as np
from sklearn.linear_model import LinearRegression
  • Step 2: Preparing the Data (The "Gotcha")
    Concept: This is where 90% of beginners get stuck. Scikit-Learn is designed for massive datasets, so it expects your inputs (X) to look like a spreadsheet (rows and columns), not a flat list. We use .reshape(-1, 1) to stand our flat list upright into a single vertical column.
    Action: 
    X = np.array([1, 2, 3, 4, 5]).reshape(-1, 1)
y = np.array([3.1, 4.9, 7.2, 8.9, 11.1])
  • Step 3: Building the Empty Brain (LinearRegression())
    Concept: Before we can train an AI, we have to bring it into existence. This line creates a "blank slate" model. It knows how to learn a line, but hasn't seen our data yet.
    Action: 
    model = LinearRegression()
  • Step 4: Training the Brain (.fit())
    Concept: "Fitting" is just the technical word for teaching. We feed our inputs (X) and our known answers (y) into the model. The algorithm does all the heavy calculus behind the scenes instantly.
    Action: 
    model.fit(X, y)
  • Step 5: Reading the Results (coef_ and intercept_)
    Concept: The model stores its final answers in special variables. coef_ (coefficient) is our slope/weight, and intercept_ is our starting bias. Because coef_ is designed to hold multiple slopes for complex 3D math, it returns a list. We use [0] to grab the first and only slope in that list.
    Action: 
    print(f"Slope: {model.coef_[0]}")
print(f"Intercept: {model.intercept_}")
Waiting for execution...

[ FROM SCRATCH ]

Calculate the mean of X and Y, the covariance, and the variance to manually derive the weight (w) and bias (b) without external ML libraries.

Incoming Transmission...

To manually derive the model without ML libraries, utilize NumPy array operations.

  • Mean: Use np.mean(array) to find the mean of X and y.
  • Slope (w): Calculate the covariance divided by the variance. 
    w = np.sum((X - X_mean) * (y - y_mean)) / np.sum((X - X_mean)**2)
  • Intercept (b): Plug the slope back into the equation. 
    b = y_mean - w * X_mean

Calculate the means first, plug them into the slope formula, then solve for the intercept.

Decryption Complete...

The Governing Formula:
w = Σ(X - X_mean) * (y - y_mean) / Σ(X - X_mean)²

Note: Don't let the math scare you. The Greek letter Sigma (Σ) just means "Sum everything up". We will make Python do this for us!

  • Step 1: The Math Toolbox (import)
    Concept: Python alone is just a blank workbench. To do heavy math, we have to bring in a toolbox. numpy (Numerical Python) is the ultimate math toolbox. We import it as np just to save typing.
    Action: import numpy as np
  • Step 2: The Data (array)
    Concept: An array is simply a neat, organized list of numbers that our math toolbox knows how to read.
    Action:
    X = np.array([1, 2, 3, 4, 5])
    y = np.array([3.1, 4.9, 7.2, 8.9, 11.1])
  • Step 3: Finding the Center (mean)
    Concept: A linear regression line rotates around the exact center of our data. We need to find the average (mean) of our inputs and outputs.
    Action:
    X_mean = np.mean(X)
    y_mean = np.mean(y)
  • Step 4: The Spread (Understanding Variance & Standard Deviation)
    Concept: Before we calculate the slope, we need to understand how the data behaves. Standard Deviation measures how spread out the numbers are from the center. Variance is just the standard deviation squared. The bottom of our formula Σ(X - X_mean)² calculates this variance!
  • Step 5: Translating Math to Code (The Numerator & Denominator)
    Concept: Look at the formula at the top. We can write that exact equation in Python. np.sum() is our Sigma (Σ) symbol!
    Action: 
    numerator = np.sum((X - X_mean) * (y - y_mean))
denominator = np.sum((X - X_mean)**2)
  • Step 6: Calculate Slope (w) and Intercept (b)
    Concept: Now just divide the top by the bottom to get the weight (slope). To find the bias (intercept), subtract the slope * the average X from the average Y.
    Action: 
    w = numerator / denominator
b = y_mean - (w * X_mean)
print(f"Slope: {w}")
print(f"Intercept: {b}")
Waiting for execution...