Give a model a raw Unix timestamp — 1757145600 — and ask it to predict food-delivery time. It will find almost nothing. Give it hour_of_day = 20, is_weekend = true, is_festival = true and the accuracy jumps, because those are the things that actually make dinner arrive late.
Same information. Completely different usefulness. That gap is feature engineering, and it moves models more than switching algorithms usually does.
What it actually is
Feature engineering is encoding domain knowledge that the algorithm cannot discover on its own.
A model can learn that larger values of a column mean higher rent. It cannot learn, from a timestamp alone, that Indian food orders spike at 8pm and that Diwali week is different — unless you give it those columns. You know that; the algorithm doesn't.
This is why a domain expert with a simple model frequently beats a pure ML specialist with a sophisticated one. The knowledge is the advantage.
Scaling
Consider two features: salary (20,000–200,000) and age (20–60). Any algorithm that measures distance will compute distances that are effectively salary alone — age contributes almost nothing, regardless of how predictive it is.
| Method | Does | Use when |
|---|---|---|
| StandardScaler | mean 0, standard deviation 1 | the default; roughly normal distributions |
| MinMaxScaler | squeezes into 0–1 | you need a bounded range; no extreme outliers |
| RobustScaler | uses median and IQR | outliers are present and you can't remove them |
| Log transform | compresses a long right tail | skewed data — income, prices, counts |
Who needs scaling: KNN, SVM, K-Means, PCA, neural networks, and any linear model with regularisation. Who doesn't: decision trees, random forests and gradient boosting — they split on thresholds, so scale is irrelevant to them.
💡 That last point is a small interview win. Being asked "do you need to scale for XGBoost?" and answering "no, trees split on thresholds so scale doesn't matter" shows you know why rather than having memorised a preprocessing checklist.
Encoding categories
Models need numbers, and how you convert a category matters enormously.
LABEL ENCODING city: Mumbai=0, Delhi=1, Chennai=2
✗ for nominal categories — it invents an order, and a linear or
distance-based model will read Chennai as "greater than" Delhi
✓ for ORDERED categories: small=0, medium=1, large=2
ONE-HOT ENCODING city_Mumbai, city_Delhi, city_Chennai (0/1)
✓ the default for nominal categories with few distinct values
✗ explodes with high cardinality — 5,000 pincodes = 5,000 columns
TARGET ENCODING replace each category with the mean target
for that category
✓ handles high cardinality well
⚠ LEAKS BADLY if computed on the full data — must be fitted on
training data only, ideally within cross-validation foldsThe rule that answers the interview question: one-hot for nominal, label or ordinal for genuinely ordered, target encoding for high cardinality — fitted on training data only.
One practical detail people miss: at prediction time you will meet a category the model never saw. Set handle_unknown="ignore" or your production pipeline will throw on the first new city.
Missing values
The instinct is to fill the gap and move on. The better first question is why is it missing?
On a loan application, "income not provided" may predict default better than any income value would. The missingness itself is signal, and filling it with a mean destroys that signal permanently.
from sklearn.impute import SimpleImputer
# Impute AND keep the information that it was missing
imp = SimpleImputer(strategy="median", add_indicator=True)
X_train_imp = imp.fit_transform(X_train) # fit on TRAIN only
X_test_imp = imp.transform(X_test) # apply the same values| Strategy | Use when |
|---|---|
| Median + indicator column | the sensible default for numeric columns |
| Mean | roughly symmetric data with no outliers |
| Most frequent / a "Missing" category | categorical columns |
| Forward fill | time series, where the last known value is a fair guess |
| Drop the column | mostly empty, or you established it carries nothing |
| Drop the rows | very few affected and the data is plentiful |
Never fill an age with zero. A zero-year-old is a real, extreme value that drags the mean and distorts every distance calculation. Missing is not zero.
Creating features
The highest-value part, and the part that needs you rather than the library.
| From | Create |
|---|---|
| A timestamp | hour, day-of-week, is_weekend, is_holiday, days-to-festival, month |
| Two dates | days between — tenure, days since last order, booking lead time |
| Two amounts | ratios — EMI-to-income, discount share, avg item value |
| An id (never use it raw) | aggregates — this restaurant's historical late rate, this customer's avg order |
| Two locations | distance, and the historical median time for that pair |
| A history | rolling mean, trend, count in the last 7/30/90 days |
Ratios deserve special mention. A ₹40,000 EMI means nothing on its own; ₹40,000 against ₹50,000 income means a great deal. Models can technically learn interactions, but handing them the ratio directly usually works far better with far less data.
Wait — this is where leakage hides
Feature engineering is the single most common source of data leakage, in two distinct forms. Both produce a model that looks brilliant and fails in production.
1. Preprocessing before splitting. Any transformation that learns something — a mean, a scale, a set of categories, a target encoding — must be fitted on the training set only.
# ✗ THE SCALER LEARNS FROM TEST ROWS
scaler.fit_transform(df)
X_train, X_test = train_test_split(df)
# ✓ SPLIT FIRST, FIT ON TRAIN
X_train, X_test = train_test_split(df)
scaler.fit(X_train)
X_test_scaled = scaler.transform(X_test)
# ✓✓ BETTER — make it structural, not something you must remember
model = Pipeline([("pre", preprocessor), ("clf", LogisticRegression())])
model.fit(X_train, y_train)2. Features that encode the future. The more dangerous form. Predicting churn using account_closed_date. Predicting delivery lateness using actual_pickup_time. The model is reading the answer.
The test to apply to every single feature: would this value be known, with this value, at the moment I make the prediction? If not, drop it — however predictive it looks. And for every historical aggregate, compute it strictly over data before the current row's timestamp.
🎯 Selection-round radar: "How would you handle missing values?" and "when do you need feature scaling?" are both common. For missing values, lead with "first I'd ask why it's missing, because missingness is often informative" and mention the indicator column — that answer stands out immediately. For scaling, name who needs it and who doesn't, and say why.
Reading feature importance honestly
Tree models hand you an importance score per feature, and it is routinely over-read.
It tells you what the model relied on — which is correlation, not causation. Ice-cream sales predict drowning incidents; neither causes the other, and both follow summer.
Two further cautions. With correlated features, credit gets split arbitrarily or assigned to one of them — a feature scoring zero may still be predictive, its information simply already carried elsewhere. And importance measures are algorithm-specific: a random forest and a linear model can rank the same features very differently.
Common mistakes
- Scaling or imputing before splitting.
- Label-encoding nominal categories, inventing a false order.
- Filling missing values with zero.
- Losing the information that a value was missing.
- Feeding raw ids to the model instead of aggregates over them.
- Including features unavailable at prediction time.
- Computing historical aggregates over data after the prediction point.
- Reading feature importance as causation.
Quick recap
| Concept | One-liner |
|---|---|
| Feature engineering | encoding domain knowledge the algorithm can't discover |
| Scaling | needed by distance-based and regularised models; not by trees |
| Encoding | one-hot for nominal, ordinal for ordered, target for high cardinality |
| Missing values | ask why first; impute AND add an indicator |
| Creating features | dates → parts, ids → aggregates, amounts → ratios |
| Leakage | fit transforms on train only; never use future information |
| The leakage test | would this value be known, with this value, at prediction time? |
Practice Zone — PYQs from real selection rounds
Six MCQs, then two tasks: engineer features for a real prediction problem, and fix a leaking preprocessing pipeline.
Why does feature scaling matter for algorithms like KNN and SVM?
Asked in
When should you use one-hot encoding rather than label encoding?
Asked in
A column has 30% missing values. What is the best first step?
Asked in
Which is an example of good feature engineering?
Asked in
How should imputation be handled when you have train and test sets?
Asked in
Which statement about feature importance is most accurate?
Asked in
Hands-on tasks:
You are predicting whether a food-delivery order will be late. Raw columns: order_timestamp, restaurant_id, customer_pincode, restaurant_pincode, order_value, item_count, rider_id. Propose engineered features and explain each.
Asked in
Find four problems in this preprocessing code and write the corrected version.
Asked in
df["city"] = LabelEncoder().fit_transform(df["city"]) # 12 cities
df["salary"] = df["salary"].fillna(df["salary"].mean())
df["age"] = df["age"].fillna(0)
X = StandardScaler().fit_transform(df.drop("target", axis=1))
X_train, X_test, y_train, y_test = train_test_split(X, y)FAQ
Doesn't deep learning make feature engineering unnecessary?
For unstructured data — images, audio, text — largely yes, and that is a genuine achievement. For tabular business data it is still where most of the gains are, because the useful features encode domain facts (festivals, business rules, ratios) that no amount of network depth invents from a timestamp.
Should I remove correlated features?
For linear models, yes — multicollinearity destabilises coefficients and destroys interpretation (lesson 3). For tree-based models it matters much less for prediction, though it does split feature importance between them and make that harder to read.
How many features is too many?
It depends on your row count more than on an absolute number — the risk is too many features relative to rows, which invites overfitting. If you have far more columns than you have rows, use regularisation (Lasso), feature selection, or dimensionality reduction (lesson 8).
Next lesson: finding structure when nobody labelled anything — Lesson 8: Clustering & PCA →