Every box you have drawn so far in this course will, eventually, fail — a server crashes, a network link drops, a downstream service times out. The question every senior interviewer eventually asks about every box on your diagram is simple and unavoidable: what happens when THIS one dies?
Timeouts and retries with backoff
Every network call needs a timeout — without one, a single hung downstream call can tie up a caller's resources indefinitely. When a call fails, retrying makes sense for genuinely transient errors, but retrying with FIXED intervals (1s, 2s, 4s) has a subtle danger: if many clients fail at the same moment (say, during an outage), they will all retry again at the exact same instant, creating a synchronized spike that can prevent the very recovery they're waiting for. Adding jitter — small randomness — spreads retries out over time instead.
Circuit breakers
Retrying every failed call against an ALREADY-struggling downstream service adds more load exactly when it can least handle it. A circuit breaker tracks recent failures and, past a threshold, "opens" — failing fast (often with a fallback) instead of piling more requests onto a service that's clearly in trouble. It periodically lets a single trial request through ("half-open") to check for recovery before fully "closing" again, giving the downstream service real room to recover rather than being hammered continuously.
Graceful degradation
When a non-critical dependency fails — say, a "recommended for you" service on an e-commerce homepage — the right response is not to fail the ENTIRE page. Graceful degradation means the rest of the page still works normally, with the broken section either hidden or replaced by a simpler fallback (a generic "trending" list instead of personalized picks). The core experience (browsing, buying) stays intact even while one non-essential piece is down.
RPO, RTO, and disaster recovery
RPO (Recovery Point Objective) answers "how much data can we afford to lose?" — if backups run every 6 hours, your RPO is up to 6 hours of data. RTO (Recovery Time Objective) answers "how long can we afford to be down?" — if failover takes 30 minutes, your RTO is 30 minutes. These numbers, set by the BUSINESS (not engineers alone), directly decide the architecture: a low RPO needs continuous replication rather than periodic backups; a low RTO needs a hot standby rather than a cold restore-from-backup process.
Single points of failure
A single point of failure (SPOF) is any ONE component whose failure takes down the whole system, no matter how redundant everything else is. Many app servers behind a load balancer don't help if that one load balancer dies, or if there's only one database behind them all. Removing a SPOF usually means adding redundancy at exactly that layer — an active-passive load balancer pair, a replicated database with automatic failover.
Common mistakes
- Retrying with fixed intervals and no jitter, risking synchronized retry storms.
- Retrying non-idempotent operations without an idempotency key, risking duplicate side effects.
- Failing an entire page when only one non-critical dependency is down.
- Setting RPO/RTO targets uniformly high "just in case" without pricing in the real infrastructure cost.
- Overlooking a SPOF because "the app servers are redundant," while the load balancer or database in front of them isn't.
Quick recap
| Concept | One-liner |
|---|---|
| Jitter | Randomizes retry timing so many clients don't retry in sync. |
| Circuit breaker | Fails fast after a failure threshold, giving a struggling service room to recover. |
| Graceful degradation | A non-critical failure shrinks functionality, not the whole page. |
| RPO | How much data loss is acceptable. |
| RTO | How much downtime is acceptable. |
| SPOF | Any single component whose failure takes down everything. |
Practice Zone
Five MCQs, then two applied questions.
A service calls a downstream payment service that has started timing out on every request. Why is a circuit breaker better than just retrying every failed call?
Asked in
Why add jitter (small randomness) to exponential backoff when retrying failed requests, instead of retrying at exact fixed intervals like 1s, 2s, 4s?
Asked in
In disaster recovery planning, what is the difference between RPO (Recovery Point Objective) and RTO (Recovery Time Objective)?
Asked in
An e-commerce homepage's 'recommended for you' section depends on a recommendation service that's currently down. What is graceful degradation in this context?
Asked in
A system has one load balancer, in front of many app servers, in front of one database. Where is the single point of failure (SPOF)?
Asked in
Your checkout service calls an inventory service that occasionally times out under load. Design the failure-handling strategy: what do you retry, how, and when do you stop retrying and do something else?
Asked in
A company runs (1) a stock trading ledger and (2) an internal wiki for engineering docs. Propose reasonable RPO and RTO targets for each, and justify the gap between them.
Asked in
FAQ
Should every operation be retried automatically?
No — only operations that are safe to repeat (naturally idempotent, or protected by an idempotency key) should be retried automatically; blindly retrying a non-idempotent write can cause real damage like duplicate charges.
Is a circuit breaker the same as rate limiting?
No — rate limiting protects YOUR service from too many incoming requests; a circuit breaker protects a downstream service (and your own caller) from continuing to hammer something that's already failing.
How do I decide RPO/RTO targets in an interview?
Tie them to the actual business cost of data loss and downtime for that specific feature — a payments ledger justifies near-zero RPO/RTO; an internal wiki does not.