Multiple Linear Regression

When predicting outcomes based on multiple factors, a single line isn't enough. We must fit a multidimensional plane through our data. Instead of one slope, we now have a weight for each input feature, and the model learns how each factor contributes to the final prediction.

y = w₁x₁ + w₂x₂ + b
w₁ = Weight 1
w₂ = Weight 2
b = Intercept (Bias)
With multiple features, the formal Hypothesis Function expands to:
hθ(x) = θ0 + θ1x1 + θ2x2
In advanced matrix notation, this is elegantly compressed to just hθ(x) = θTx.
DATA POINTS: [ 40 ]
ACTIVE EQUATION: [ y = 1.0x₁ + 1.0x₂ + 0.0 ]
MSE: [ --- ]
J(θ) = (1 / 2m) Σ (hθ(x(i)) - y(i))² The goal of our model is to minimize this Cost Function J(θ).
  Weight 1 (w₁) 1.0
  Weight 2 (w₂) 1.0
  Intercept (b) 0.0
Loading Python Environment...

[ APPLIED MULTIPLE LINEAR REGRESSION ]

Use Scikit-Learn to fit a plane to the provided 2-feature X matrix and y vector. Print the weights and intercept.

Incoming Transmission...

Import LinearRegression from sklearn.linear_model. Instantiate the model and pass the 2D X matrix and y vector into .fit(). Because X is already 2D (a list of lists), no .reshape() is needed!

model = LinearRegression()
model.fit(X, y)

The learned parameters are in model.coef_ (an array of weights, one per feature) and model.intercept_ (the bias).

Decryption Complete...

The Scikit-Learn Workflow:

Note: This is the same workflow as Simple Linear Regression, but now our input X has multiple columns — one for each feature.

  • Step 1: The Toolboxes (import)
    Concept: We bring in our math toolbox (numpy) and our machine learning toolbox (sklearn).
    Action: 
    import numpy as np
from sklearn.linear_model import LinearRegression
  • Step 2: The Matrix (No Reshape Needed!)
    Concept: Unlike simple regression where X was a flat list, here X is already a matrix — each row is a sample and each column is a feature. If we write it as a list of lists, Scikit-Learn is happy.
    Action: 
    X = np.array([[1, 2], [2, 4], [3, 5], [4, 4], [5, 7]])
y = np.array([5.1, 10.2, 14.5, 12.8, 19.7])
  • Step 3: Building the Empty Brain
    Concept: We create a blank model that now knows how to learn a plane instead of a line.
    Action: model = LinearRegression()
  • Step 4: Training the Brain
    Concept: We feed our multi-column X and y into model.fit(). The algorithm figures out a separate weight for each column.
    Action: model.fit(X, y)
  • Step 5: Multiple Weights!
    Concept: model.coef_ now returns an array of weights — one for each feature column. The first element is w₁ and the second is w₂. The intercept is still a single number.
    Action: 
    print(f"Weights: {model.coef_}")
print(f"Intercept: {model.intercept_}")
Waiting for execution...

[ FROM SCRATCH — THE NORMAL EQUATION ]

Use NumPy matrix algebra (The Normal Equation) to calculate the weights directly. No ML library allowed!

Incoming Transmission...

The Normal Equation solves for all weights at once using matrix algebra:

W = (Xᵀ X)⁻¹ Xᵀ Y

Use np.linalg.inv() for the matrix inverse and .T for the transpose. Don't forget to add a column of 1s to X to account for the intercept!

Decryption Complete...

The Normal Equation:
W = (Xᵀ X)⁻¹ Xᵀ Y

Note: With multiple features, basic division fails. We use matrix algebra to solve for all weights simultaneously. Think of it as solving a system of equations in one shot.

  • Step 1: The Setup
    Concept: Import numpy and define your X matrix (2 features) and y vector.
    Action: 
    import numpy as np
X = np.array([[1, 2], [2, 4], [3, 5], [4, 4], [5, 7]])
y = np.array([5.1, 10.2, 14.5, 12.8, 19.7])
  • Step 2: The Bias Column (The Trick)
    Concept: To calculate the intercept (b) at the same time as the weights, we add a column of 1s to our X matrix. This "absorbs" the bias into the matrix math so we can solve for everything in one equation.
    Action: 
    X_b = np.c_[np.ones((len(X), 1)), X]
  • Step 3: The Transpose
    Concept: Flipping a matrix on its side (swapping rows and columns) is called transposing. If X has 5 rows and 3 columns, its transpose has 3 rows and 5 columns. We need this for the matrix multiplication to work.
    Action: X_T = X_b.T
  • Step 4: The Inverse & Dot Product
    Concept: The .dot() method multiplies matrices together. np.linalg.inv() finds the inverse (think of it as "dividing" matrices). We chain these together following the formula from top to bottom.
    Action: 
    weights = np.linalg.inv(X_T.dot(X_b)).dot(X_T).dot(y)
  • Step 5: Reading the Answer
    Concept: The resulting weights array contains all our values. The first number is the intercept (b), because we added the 1s column first. The rest are the feature weights (w₁, w₂).
    Action: 
    print(f"Intercept (b): {weights[0]}")
print(f"Weight 1 (w1): {weights[1]}")
print(f"Weight 2 (w2): {weights[2]}")
Waiting for execution...