A broker in your area can look at a flat and quote a rent within a couple of thousand rupees. Ask how, and you get "experience" — but push a bit and it comes out as arithmetic: roughly ₹40 per square foot, a bit less far from the metro, a bit more with a lift.
That is linear regression, done in a human head. This lesson makes it explicit — and covers the one whose name lies about what it does.
Linear regression
Plot rent against area for a few hundred flats and you get a cloud of points sloping upward. Linear regression draws the straight line through that cloud that fits best — and "best" has a precise meaning: the line that minimises the sum of the squared vertical distances from the points to the line.
Squared, for two reasons worth knowing. It makes every error positive so misses above and below don't cancel out. And it punishes one big miss far more than several small ones, which is often what you want — though not always, as the metrics section shows.
model = LinearRegression().fit(df[["area_sqft"]], df["rent"])
print("slope :", model.coef_[0]) # rupees per square foot
print("intercept:", model.intercept_) # the baselineResult
Nobody told it 41.8. It found the number that made the line fit — and the number is readable, which is why linear regression survives despite being the simplest thing in the toolbox.
More than one feature
Add bedrooms, distance to the metro, floor number, and the line becomes a plane in more dimensions — but nothing conceptual changes. Each feature gets its own coefficient, and each coefficient means "the effect of this feature, holding everything else constant".
features = ["area_sqft", "bedrooms", "distance_metro_km", "has_lift"]
model = LinearRegression().fit(df[features], df["rent"])
for name, coef in zip(features, model.coef_):
print(f"{name:20s} {coef:>10.2f}")Result
Read those: ₹42 per extra square foot, ₹310 less per kilometre from the metro, ₹1,200 more with a lift. Sensible — except bedrooms is negative, which says more bedrooms means lower rent. That is not a data-entry error, and the "assumptions" section below explains it.
Logistic regression — the misleading name
Logistic regression is a classification algorithm. Despite the name. This trips people up constantly and is asked deliberately for that reason.
The mechanism: it computes the same kind of weighted sum as linear regression, then squashes the result through a sigmoid function that maps any number into the range 0 to 1. That gives you a probability. A threshold — 0.5 by default — turns the probability into a class.
from sklearn.linear_model import LogisticRegression
model = LogisticRegression().fit(X_train, y_train)
model.predict(X_new) # -> array([1]) the class
model.predict_proba(X_new) # -> [[0.13, 0.87]] P(no), P(yes)💡 In production, use the probability, not the class. A retention team wants customers ranked by churn risk so they can call the top 200 — a bare yes/no throws that information away. And the threshold is a free dial you can tune from business economics without retraining anything (lesson 6).
Its virtues: fast, needs little data, and the coefficients are interpretable — which is why banks and insurers still use it where decisions must be explained.
How to measure a regression
| Metric | Means | Use when |
|---|---|---|
| MAE | average absolute error, in the target's own units | every unit of error costs the same; outliers shouldn't dominate |
| RMSE | square root of mean squared error — punishes big misses | one large miss is much worse than several small ones |
| R² | proportion of variance explained; 0 = no better than the mean | a quick sense of fit; never as the only number |
| MAPE | average percentage error | values span orders of magnitude and relative error matters |
MAE is in rupees, which is what a business person understands. RMSE is also in rupees but weighted toward large errors. R² has no units and is easy to over-read — note especially that R² never decreases when you add a feature, even a useless one, which is why adjusted R² exists.
Choose the metric that matches the cost of being wrong. Predicting hospital bed demand? One catastrophic underestimate matters more than many small ones — RMSE. Predicting daily sales where a few festival days are 10× normal? RMSE would chase those and ruin the other 350 days — MAE, plus a festival feature.
Wait — why is my bedrooms coefficient negative?
Back to that ₹-890 per bedroom. More bedrooms lowering rent is obviously wrong, and yet the model is behaving correctly. The cause is multicollinearity: two features carrying nearly the same information.
Bigger flats have more bedrooms. Once area_sqft is in the model, bedrooms adds almost nothing independent — so the algorithm has no stable way to split the credit between them, and the coefficient can swing wildly or flip sign. Note it was also not statistically significant, which is the tell.
Multicollinearity usually leaves predictions fine and destroys interpretation. That matters enormously when someone asks "what drives rent?" and not at all when you only need a number.
Fixes: drop one of the correlated features, or replace both with something independent — area_sqft plus bedrooms_per_1000sqft. Diagnose it with a correlation matrix or the variance inflation factor.
🎯 Selection-round radar: two questions recur. "Is logistic regression classification or regression?" → classification; it outputs a probability which a threshold turns into a class. And "when would you use MAE over RMSE?" → when outliers shouldn't dominate, because RMSE squares errors and a few large misses swamp it.
Ridge and Lasso
When a linear model has many features relative to rows, it can fit noise — the coefficients grow large and specific to the training data, and test performance collapses. Regularisation adds a penalty on coefficient size to prevent that.
| Ridge (L2) | Lasso (L1) | |
|---|---|---|
| Penalty on | squared coefficients | absolute coefficients |
| Effect | shrinks all coefficients toward zero | can drive some to exactly zero |
| Side benefit | handles correlated features gracefully | performs feature selection for you |
| Use when | features are correlated; keep them all, reduced | you suspect many features are useless |
The one-line distinction interviewers want: Lasso can zero out features; Ridge shrinks but keeps them. ElasticNet combines both. Lesson 5 covers regularisation as a general idea.
Common mistakes
- Calling logistic regression a regression algorithm.
- Reporting R² from the training set as the model's performance.
- Interpreting coefficients without checking for multicollinearity.
- Using RMSE on data with legitimate extreme values you could model instead.
- Adding features to raise R² — it never decreases, so this proves nothing.
- Reading a coefficient as proof of causation.
- Forgetting that linear models need scaling when regularisation is used.
Quick recap
| Concept | One-liner |
|---|---|
| Linear regression | the line minimising squared vertical distances to the points |
| Coefficient | effect of this feature holding the others constant |
| Logistic regression | classification — sigmoid gives a probability, a threshold gives a class |
| MAE vs RMSE | equal-cost errors vs big misses being much worse |
| R² | variance explained; never decreases with more features |
| Multicollinearity | correlated features destabilise coefficients; predictions may be fine |
| Ridge vs Lasso | shrink everything vs zero some out (feature selection) |
Practice Zone — PYQs from real selection rounds
Six MCQs, then two tasks: interpret a real regression output, and pick the right setup and metric for three problems.
What does linear regression actually find?
Asked in
Despite its name, logistic regression is used for:
Asked in
What does R² (R-squared) tell you?
Asked in
When would you prefer MAE over RMSE as an error metric?
Asked in
What is multicollinearity and why does it matter?
Asked in
Your linear regression has R² = 0.95 on training data and 0.42 on test data. What is happening?
Asked in
Hands-on tasks:
You fit a model predicting monthly rent in ₹. Interpret this output and say what you would investigate.
Asked in
Feature Coefficient p-value
------------------------------------------------
area_sqft 42.10 0.000
bedrooms -890.00 0.212
distance_metro_km -310.50 0.001
floor_number 95.20 0.043
has_lift 1200.00 0.008
intercept 4500.00
R² = 0.78 Adjusted R² = 0.77
Train RMSE = 2,100 Test RMSE = 5,800For each, choose the model type and the error metric, with a reason: (1) predicting delivery time in minutes where being 40 minutes late is far worse than four times being 10 minutes late, (2) predicting whether a customer will renew, (3) predicting daily sales where a few festival days are 10× normal.
Asked in
FAQ
What if the relationship isn't a straight line?
Add transformed features — area squared, or log(area) — and a linear model can fit a curve, because it is linear in its coefficients, not necessarily in the raw features. Beyond mild curvature, a tree-based model handles non-linearity without you having to guess the shape.
Does a large coefficient mean an important feature?
Not on unscaled data. A coefficient of 42 for square feet and 1,200 for has_lift are not comparable, because the features are measured on completely different scales. Standardise the features first if you want to compare magnitudes — and even then, importance is not causation.
When would I use linear regression over a gradient-boosted model?
When you need interpretability — a regulator, a client, or a colleague who has to defend the decision. Also when data is small, when the relationship really is roughly linear, or as a baseline, which you should fit regardless so you know what the complicated model actually bought you.
Next lesson: the algorithms that predict categories, and how to choose between them — Lesson 4: Classification Algorithms →