Decision Frameworks & Tradeoff Matrices

A system design interview is rarely testing whether you know what a cache is. It's testing whether you can look at a set of constraints — read/write ratio, consistency requirements, team size, latency budget — and pick the right tool, then defend why. This chapter collects the recurring decisions and gives you a repeatable way to reason through each one, instead of memorizing "the answer."

Every other chapter on this site dives deep into one mechanism. This one is different: it's a map of the decision points themselves — the moments in a design where you have to choose a direction and justify it out loud.

The Meta-Framework

Before any specific tradeoff, run every decision through the same four questions:

1. What does this optimize for, and what does it cost? Every technique trades one property for another — there is no free scaling.
2. What's the failure mode if I'm wrong? Some wrong choices are cheap to reverse (a cache TTL), others are catastrophic (a shard key).
3. What's the simplest thing that satisfies today's requirements? You can almost always add complexity later; removing it is much harder.
4. Would this decision still make sense at 10x scale? At 100x? Not to over-engineer for 100x, but to know where the seams are so you can cut there later.

              ┌───────────────────────────┐
              │   New design decision      │
              └─────────────┬─────────────┘
                             ▼
        1. What does it optimize for, what does it cost?
                             ▼
        2. What's the failure mode if this choice is wrong?
                             ▼
        3. What's the simplest thing that meets today's needs?
                             ▼
        4. Does it still hold at 10x / 100x scale?
                             ▼
              ┌───────────────────────────┐
              │  "It depends on X: if X,   │
              │   A; if not-X, B" — stated │
              │   out loud, with the cost  │
              └───────────────────────────┘

SQL vs NoSQL

Decision rule: if your access patterns are known in advance and write throughput/schema flexibility dominate, lean NoSQL. If you need multi-row transactions, ad-hoc reporting, or strong consistency guarantees, lean SQL. Most real systems use both — SQL for the transactional core, NoSQL (or a cache) for derived, high-volume, read-heavy data. See Databases and Database Scaling for the mechanics.

REST vs GraphQL vs gRPC

Decision rule: public-facing, cacheable, resource-oriented API → REST. Multiple frontends with very different data needs from the same backend → GraphQL. High-throughput internal microservice communication where both ends are your own code → gRPC. Full breakdown in APIs.

Synchronous vs Asynchronous Processing

• Use synchronous (request/response) when the caller cannot proceed without the answer — validating a payment before confirming an order, checking auth before serving a page.
• Use asynchronous (queue/event) when the work can happen after the caller has already gotten an acknowledgment — sending a confirmation email, generating a thumbnail, updating a search index, billing reconciliation.

SYNCHRONOUS
Client → Server: place order
                  Server: charge card, reserve inventory, write order  (all inline)
Client ← Server: 200 OK (order confirmed)                — client waited for everything

ASYNCHRONOUS
Client → Server: place order
                  Server: charge card, write order            (only what's required now)
Client ← Server: 202 Accepted (order received)             — client freed immediately
                  Server → Queue: send-confirmation-email, update-search-index, ...
                  Workers: process queue in the background, retry on failure

The tell-tale symptom of "this should have been async" is a request handler doing meaningfully more work than the client is waiting to see the result of. See Message Queues for the mechanics of making this durable.

Push vs Pull

Decision rule: if updates are frequent and low-latency delivery matters (chat, live scores, presence), push. If updates are rare, latency tolerance is high, or the consumer might be offline for long stretches (webhooks with retries, batch sync), pull — or push with a durable queue behind it as a hybrid. See Real-Time & Communication.

Consistency vs Availability (in practice)

This is CAP theorem applied to an actual product decision, not the theorem itself (see CAP Theorem for that). The practical question: when the network partitions, or a replica is temporarily behind, what do you tell the user?

• Choose consistency (CP) for anything involving money, inventory counts, or uniqueness constraints (usernames, seat bookings) — showing a stale-but-available answer here causes real harm (double-spend, oversold seats).
• Choose availability (AP) for anything where a slightly stale view is harmless and no answer is worse than an old answer — social feeds, view counts, "last seen" timestamps, product recommendations.

Rule of thumb: ask "what happens if the user acts on stale data?" If the answer is "nothing bad," go AP. If the answer is "we lose money or violate an invariant," go CP.

Cache-Aside vs Write-Through vs Write-Behind

Decision rule: cache-aside is the safe default for read-heavy workloads. Write-through when you need the cache and DB to never disagree. Write-behind only when write throughput is the bottleneck and you can tolerate a small durability window — pair it with a durable queue, not a bare in-memory cache. Full detail in Caching.

Build vs Buy (Managed Service vs Self-Hosted)

Ask, in order:

1. Is this core to our competitive advantage? If yes, and only if yes, consider building — differentiation is the only reason to accept the operational burden.
2. Does a managed offering meet our latency/compliance/cost constraints? Most of the time, for infrastructure primitives (queues, caches, search, auth), the answer is yes.
3. Can our team actually operate this reliably at 3am? Self-hosting Kafka, Elasticsearch, or a distributed cache is a genuine operational commitment, not a one-time setup cost.

Rule of thumb: buy/use-managed for anything that isn't your product's differentiator. Build only where owning the internals gives you a real, defensible advantage.

Monolith vs Microservices, Vertical vs Horizontal Scaling

These deserve full chapters rather than a table row — see Architecture Patterns and Horizontal vs Vertical Scaling. The one-line versions:

• Start monolithic. Split into services when a specific, felt pain (independent deploy cadence, independent scaling, team ownership boundaries) exceeds the operational cost of distribution — not preemptively.
• Scale vertically first for anything stateful or single-threaded-bottlenecked; scale horizontally once you've hit hardware ceilings or need fault tolerance through redundancy, not just more capacity.

Interview Tips

• Never present a tradeoff as "X is better." Present it as "X optimizes for A at the cost of B; here's why A matters more for this system."
• Anchor every tradeoff decision to a number where you can: "if writes are 100x reads, I'd shard early" beats "sharding might help."
• When the interviewer pushes back on your choice, that's usually not a signal you're wrong — it's an invitation to state the tradeoff you made explicitly. Naming the cost you accepted is often worth more than defending the choice.
• If you're ever unsure, say what you'd measure to decide, and what threshold would flip your answer. That's a stronger answer than a confident guess.