"Make it faster" and "make it handle more users" sound like the same request. They are not — and confusing them is one of the fastest ways to sound like you're guessing in an HLD interview. A single toll booth that processes each car in 3 seconds flat (fast!) can still create a two-hour traffic jam if 10,000 cars arrive in the same minute (low throughput). This lesson pins down four words interviewers use precisely, even when everyday speech doesn't.
Latency vs throughput
Latency is how long ONE request takes, from the moment it's sent to the moment its response comes back — measured in milliseconds. Throughput is how many requests the whole system can handle per second, across ALL users combined. A system can have excellent latency and terrible throughput (each request is answered in 5ms, but only 10 requests per second can be handled before everything queues up) or the reverse (handles 50,000 requests per second in aggregate, but each individual one takes a noticeably slow 800ms).
Why averages lie: P50, P95, P99
Say 99 users get a response in 50ms, and 1 unlucky user (maybe their request hit a server doing garbage collection, or a slow database query) waits 5 seconds. The AVERAGE latency looks fine — a little under 100ms — but that hides the fact that 1% of your users are having a genuinely bad experience. This is why real systems report percentiles instead of averages: P50 (median — half of requests are faster than this) tells you about the typical case, while P95 and P99 (95% and 99% of requests are faster than this) tell you about the worst experiences a meaningful chunk of your users actually have.
Availability vs reliability
These two get conflated constantly, but they answer different questions. Availability asks: is the system UP and responding right now? Reliability asks: does the system correctly do what it's supposed to do, over time, without producing wrong results or silently losing data? A system can be available but not reliable — it responds instantly to every request, but sometimes returns the wrong bank balance. It can also be reliable but not available — it never returns an incorrect answer, but it's down for maintenance every night at 2 AM.
The nines of availability, and what they cost in downtime
Availability is usually stated as a percentage of "the nines" — and the jump from one nine to the next is far more expensive to build than the percentage difference suggests:
99% ('two nines') ≈ 3.65 days of downtime per year
99.9% ('three nines') ≈ 8.7 hours of downtime per year
99.99% ('four nines') ≈ 52.6 minutes of downtime per year
99.999% ('five nines') ≈ 5.3 minutes of downtime per yearGoing from 99% to 99.9% sounds like a tiny numerical change but means building automatic failover instead of hoping a human notices an outage within a day. Going further, to 99.99% or beyond, usually means multi-region active-active infrastructure — a genuinely different (and more expensive) architecture, not just "trying harder." This is exactly why a strong HLD answer states an availability TARGET early and ties later architectural decisions back to it.
Common mistakes
- Using "fast" and "scalable" interchangeably when they describe latency and throughput respectively.
- Quoting only average latency, hiding how bad the P99 tail actually is.
- Treating "the system is up" (availability) as the same as "the system is correct" (reliability).
- Promising "five nines" availability without acknowledging the very different infrastructure that requires.
Quick recap
| Concept | One-liner |
|---|---|
| Latency | Time for one request. |
| Throughput | Requests handled per second, in aggregate. |
| P99 | 99% of requests are faster than this — exposes the bad tail averages hide. |
| Availability | Is it up right now? |
| Reliability | Does it correctly do its job over time? |
Practice Zone
Five MCQs, then two applied questions.
Which statement correctly separates latency from throughput?
Asked in
Why do system designers care about P99 latency, not just average latency?
Asked in
A system promises 99.9% availability. Roughly how much downtime does that allow per year?
Asked in
A payments API responds instantly to every request (100% uptime) but occasionally returns the wrong balance due to a bug. Is this system available? Is it reliable?
Asked in
A single app server handles 500 QPS comfortably. Traffic grows to 6,000 QPS. What is the MOST direct fix, assuming the servers are stateless?
Asked in
Your monitoring shows average latency of 120ms for a checkout API, well within your 200ms SLA. Yet support tickets say 'checkout hangs sometimes.' What metric were you missing, and what would you check next?
Asked in
Roughly how much yearly downtime does 99.99% availability allow, and how does that compare to 99.9%?
Asked in
FAQ
Which percentile should I always mention in an interview?
P99 is the most commonly expected one — it shows you're thinking about the worst-case experience for a meaningful slice of real users, not just the typical case.
Can a system be 100% available?
Not realistically — networks, hardware, and software all fail eventually. Real systems target a specific number of nines based on what downtime actually costs the business, not perfection.
Is higher availability always better?
Not unconditionally — each additional nine costs meaningfully more to build and operate, so the right target depends on what downtime actually costs versus what the extra infrastructure costs.