Your college fest's registration form was hosted on one laptop acting as a server, and it crashed the moment 300 students refreshed it at once. There are exactly two honest fixes: give that one laptop a much bigger CPU and more RAM, or put ten ordinary laptops behind something that spreads the 300 requests across all of them. Those two fixes have names — vertical and horizontal scaling — and almost every HLD design leans on the second one.
Vertical vs horizontal scaling
Vertical scaling means making one machine more powerful — more CPU cores, more RAM, a faster disk. It is simple: no code changes, no new components. But it has a hard ceiling (there is a biggest machine money can rent), it usually needs a restart to apply, and that one bigger machine is STILL a single point of failure — if it dies, everything dies with it.
Horizontal scaling means running the SAME application on many ordinary machines, with something in front deciding which machine handles each incoming request. It has no real ceiling — need more capacity, add more machines — and it adds redundancy for free: one machine dying doesn't take down the others. The cost is that your application now has to be designed to not care which specific machine handles which request.
Why stateless services are the precondition
Horizontal scaling only works cleanly if any server can handle any request — which means a server cannot keep important state (like "is this user logged in") only in its own local memory. If it did, a user whose next request lands on a DIFFERENT server would suddenly appear logged out. The fix: keep that state somewhere shared and reachable by every server — a database, or a shared cache like Redis — so every server is interchangeable.
A common place to keep shared session state
RedisHow a load balancer picks a server
Once you have many interchangeable servers, something has to decide which one gets each incoming request. A few common strategies:
- Round robin — cycle through servers in order, one request each. Simple, assumes every request costs roughly the same.
- Least connections — send the new request to whichever server currently has the fewest active connections. Better when request durations vary a lot.
- Weighted routing — some servers are more powerful and should get proportionally more traffic; weights encode that directly.
L4 vs L7 load balancing
A Layer 4 (transport-layer) load balancer routes based on IP address and port alone — it is fast and simple, but blind to the actual content of the request. A Layer 7 (application-layer) load balancer reads the actual HTTP request — the URL path, headers, cookies — and can route based on that, e.g. sending /api/video/* requests to a different server pool than /api/checkout/*. L7 is more flexible but does more work per request, since it has to actually parse the traffic.
Health checks
A load balancer is only useful if it stops sending traffic to a server that has died or is struggling. Health checks are periodic pings ("are you alive and responding correctly?") — a server that fails enough consecutive checks gets pulled out of rotation automatically, and traffic quietly shifts to the remaining healthy servers, until it starts passing checks again.
Sticky sessions — a necessary evil
Sometimes an application genuinely can't be made fully stateless quickly (legacy code, or state that's expensive to externalize) — "sticky sessions" is a compromise where the load balancer routes a given user's requests to the SAME server every time, using a cookie. It works, but it undermines some of horizontal scaling's benefits: that one server now holds state that would be lost if it died, and load can become uneven if many "sticky" users land on the same server. It is a pragmatic fallback, not the ideal — true statelessness is preferred whenever it's achievable.
Common mistakes
- Forgetting that horizontal scaling REQUIRES stateless services — adding servers to a stateful app just creates inconsistent behavior.
- Treating the load balancer itself as unbreakable — it needs its own redundancy (an active-passive pair), or it becomes a new single point of failure.
- Choosing round robin blindly when request durations vary wildly, where least-connections would balance load far better.
- Reaching for sticky sessions as a first choice instead of a last resort.
Quick recap
| Concept | One-liner |
|---|---|
| Vertical scaling | Bigger machine — simple, has a ceiling, still a SPOF. |
| Horizontal scaling | More machines — no real ceiling, needs statelessness. |
| Load balancer routing | Round robin, least-connections, or weighted, depending on workload. |
| L4 vs L7 | L4 = fast, blind to content; L7 = smarter, more work per request. |
| Health checks | Automatically pull dead/struggling servers out of rotation. |
Practice Zone
Five MCQs, then two applied questions.
What is the key difference between vertical and horizontal scaling?
Asked in
Why does horizontal scaling usually require 'stateless' application servers?
Asked in
Two load-balancing strategies: 'round robin' and 'least connections.' When would least-connections clearly beat round robin?
Asked in
What is one thing an L7 (application-layer) load balancer can do that an L4 (transport-layer) one cannot?
Asked in
What is the point of a load balancer's health check?
Asked in
A team scales their login service from 1 to 5 instances behind a round-robin load balancer. Users start getting logged out randomly. What's the likely cause, and what are two ways to fix it?
Asked in
An e-commerce API has two kinds of endpoints on the same fleet: GET /product/:id (fast, ~10ms) and POST /generate-invoice-pdf (slow, ~2-5 seconds). Would you use round robin or least-connections here, and why?
Asked in
FAQ
Is horizontal scaling always better than vertical?
Not always to start with — vertical scaling is simpler for a small system with modest load. Horizontal scaling earns its complexity once a single machine's ceiling becomes a real constraint.
What's the difference between a load balancer and a reverse proxy?
A reverse proxy sits in front of one or more backend servers forwarding requests; a load balancer is a reverse proxy specifically focused on distributing load across MULTIPLE backend servers — the terms overlap a lot in practice.
Can the load balancer itself become a bottleneck?
Yes — that's why production load balancers are themselves deployed redundantly (often an active-passive or DNS-based pair), so the load balancer doesn't become the new single point of failure.