Neural networks have a reputation for being mysterious, which is unfortunate, because the individual unit does three things: multiply, add, and squash. That is genuinely all of it. The interesting behaviour comes from stacking a lot of them, not from any one being clever.
This lesson gives you what placement interviews ask for — the mechanics in plain language, the vocabulary, and the judgement question that matters most: when a neural network is the wrong choice.
One neuron
inputs: x1, x2, x3 (features, or outputs of the layer before)
weights: w1, w2, w3 (learned)
bias: b (learned)
z = (x1·w1) + (x2·w2) + (x3·w3) + b # weighted sum
output = activation(z) # squash itA weighted sum plus a bias, then an activation function. The weights say how much each input matters; the bias shifts the threshold at which the neuron responds. Both are learned — nobody sets them.
If that looks like linear regression with an extra step, that's because it is. A single neuron with a sigmoid activation is logistic regression. Everything a network can do beyond that comes from combining many of them.
Why the activation function matters
This is the cleanest interview question in the lesson, because the answer is a mathematical fact rather than a rule of thumb.
Without a non-linear activation, stacking layers is pointless. A linear function of a linear function is still a linear function — so a hundred-layer network with no activations collapses into the equivalent of one linear layer, and could never learn anything logistic regression couldn't.
| Activation | Does | Used for |
|---|---|---|
| ReLU | max(0, z) — negatives become zero | the default for hidden layers; fast and effective |
| Sigmoid | squashes into 0–1 | binary classification output |
| Softmax | a probability distribution over classes | multi-class output layer |
| Tanh | squashes into −1 to 1 | older architectures; sometimes in recurrent networks |
💡 A ReLU detail that shows depth: a neuron whose input is always negative outputs zero forever and stops learning — the dying ReLU problem. LeakyReLU (a small slope for negatives) is the usual fix, and "many activations are always zero" is a real thing to check when training stalls.
Layers, and what depth buys
Neurons are arranged in layers: an input layer (one per feature), one or more hidden layers, and an output layer (one neuron for binary, one per class for multi-class, one for regression).
What depth gives you is hierarchy. In an image network, early layers learn edges; middle layers combine edges into shapes; later layers combine shapes into objects. Nobody programmed that progression — it emerges because each layer can only build from what the previous one produced.
That hierarchy is why neural networks dominate unstructured data — and, equally, why they add less on a table of thirty business columns, where there is no comparable hierarchy to discover.
How it learns
- Forward pass — data flows through the network and produces a prediction.
- Loss — compare the prediction to the true answer; the difference is a number.
- Backpropagation — work backwards from the output, computing how much each weight contributed to that error.
- Gradient descent — nudge every weight slightly in the direction that reduces the error.
- Repeat for many batches, for several epochs.
The one-line answer to "what is backpropagation?": the algorithm that assigns blame for the error to each weight, so gradient descent knows which way to move it. At placement level you need that idea, not the chain-rule derivation.
The dials
| Dial | Controls | Symptom when wrong |
|---|---|---|
| Learning rate | size of each weight update | too high: loss oscillates. too low: loss barely moves |
| Epochs | passes over the training data | too many: overfitting — use early stopping |
| Batch size | examples per weight update | too small: noisy gradients. too large: memory, and can generalise worse |
| Architecture | layers and neurons per layer | too small: underfits. too large: overfits and trains slowly |
| Dropout | fraction of neurons deactivated per training step | too little: overfits. too much: underfits |
The learning rate is usually the one that matters most, and its two failure modes are diagnosable from a loss curve in seconds — which is why you plot training and validation loss from the start rather than looking at a final number.
On dropout: it randomly deactivates a fraction of neurons during training, so the network can't depend on any single unit and learns redundant representations. The detail interviewers probe: it is active during training only — at inference the full network is used.
Wait — when should I NOT use one?
More often than students expect, and being able to say so is worth more in an interview than knowing another architecture name.
| Situation | Better choice |
|---|---|
| Tabular business data | gradient-boosted trees — usually more accurate, far less tuning |
| A few thousand rows | almost anything simpler; networks are data-hungry |
| Decisions must be explained | logistic regression or a shallow tree |
| Very tight latency or no GPU | a linear model or small tree ensemble |
| Images, audio, text, long sequences | a neural network — and ideally a pretrained one |
Neural networks dominate unstructured data; gradient-boosted trees still win on tables. That sentence is a widely reported result and a widely ignored one, and saying it confidently marks you out from candidates who reach for deep learning because it sounds advanced.
🎯 Selection-round radar: the two questions asked most are "why do you need a non-linear activation?" (because composed linear functions are still linear, so depth would buy nothing) and "what is backpropagation?" (assigning error blame backwards to each weight so they can be adjusted). If they then ask which model you'd pick for a tabular dataset, say gradient boosting — and say why.
The architecture families
| Type | Built for | Key idea |
|---|---|---|
| Feedforward (MLP) | general tabular use | the basic stacked-layers network |
| CNN | images | filters that detect local patterns anywhere in the image |
| RNN / LSTM | sequences | a hidden state carried from step to step |
| Transformer | text, and increasingly everything | attention — every position looks at every other in parallel |
The transformer is the one behind every large language model. If you want that in depth, it has its own lesson in the Generative AI course.
And the practical note that matters more than any of these names: for images, audio and text, start from a pretrained model and fine-tune it. Training from scratch needs orders of magnitude more data than you have, and the result would be worse.
Common mistakes
- Using a neural network on tabular data because it sounds advanced.
- Training from scratch when a pretrained model exists.
- Not scaling inputs — networks need it as much as KNN does.
- Setting a learning rate without watching the loss curve.
- Training for a fixed large number of epochs with no early stopping.
- Adding layers to fix underfitting when the features carry no signal.
- Forgetting that dropout is training-only.
- No baseline, so nobody knows what the network actually bought.
Quick recap
| Concept | One-liner |
|---|---|
| A neuron | weighted sum + bias, then an activation |
| Non-linearity | without it, depth is pointless — linear composed is still linear |
| Depth | builds a hierarchy — edges → shapes → objects |
| Backpropagation | assigns error blame backwards to each weight |
| Learning rate | the most important dial; oscillating or flat loss diagnoses it |
| Dropout | random deactivation during training only |
| When not to | tabular data, small data, explainability, tight latency |
Practice Zone — PYQs from real selection rounds
Six MCQs, then two tasks: decide where a neural network belongs, and debug three training runs from their loss curves.
What does a single neuron in a neural network do?
Asked in
Why is a non-linear activation function essential?
Asked in
What is backpropagation?
Asked in
What does the learning rate control?
Asked in
For a tabular business dataset with 20,000 rows and 25 features, would you use a neural network?
Asked in
What is dropout and why does it help?
Asked in
Hands-on tasks:
Decide for each and justify: (1) classifying X-ray images, (2) predicting insurance claim amounts from 30 tabular features, (3) transcribing Hindi speech, (4) predicting machine failure from 40 sensor readings with 5,000 labelled failures.
Asked in
Three neural network training runs go wrong in different ways. Diagnose each: (A) loss jumps up and down and never settles, (B) loss decreases for 3 epochs then stops changing at all, (C) training loss falls to near zero while validation loss climbs.
Asked in
FAQ
How many layers and neurons should I use?
Start small — one or two hidden layers — and grow only if the model underfits. For most problems, an existing architecture for that data type is a better starting point than a design of your own. And check first that a simpler model doesn't already solve it, because that answer is often yes.
Do I need a GPU?
Not for small feedforward networks on tabular data — a CPU is fine. You need one for images, audio and anything transformer- shaped, where training would otherwise take days. Free cloud notebooks provide a GPU, which is enough for learning and for most fine-tuning experiments.
What is transfer learning?
Starting from a model already trained on a large general dataset and adapting it to your smaller specific one. It is the default approach for images, audio and text, because the early layers have already learned general features you would otherwise need enormous data to discover. It is the same idea as fine-tuning an LLM.
Last lesson: how all of this becomes a real project that survives contact with production — Lesson 10: ML Project Lifecycle →