System Design Fundamentals Every Developer Should Know

Why System Design Matters

Feature code is written once and maintained forever. System design decisions persist longer — sometimes for the entire lifespan of a product. A poorly designed data layer can make vertical scaling impossible. A missing cache layer can make a working product unusable at scale.

These are the fundamentals every developer should internalize before they're needed in production.

1. Horizontal vs Vertical Scaling

**Vertical scaling** — add more resources (CPU, RAM) to a single machine. Simple to implement, limited ceiling, single point of failure.

**Horizontal scaling** — add more machines and distribute load. More complex, effectively unlimited ceiling, resilient to individual failures.

Most modern systems start vertical and plan for horizontal. Plan for horizontal even if you don't need it yet — architectural decisions made at 1,000 users are expensive to undo at 1,000,000.

2. Load Balancers

A load balancer sits in front of your server fleet and distributes incoming requests.


Client → Load Balancer → Server 1
                       → Server 2
                       → Server 3

**Distribution strategies:** - **Round-robin** — rotate through servers in order (simple, default) - **Least connections** — route to the server with the fewest active connections - **IP hash** — route the same client IP to the same server (useful for stateful sessions)

Health checks run continuously. When a server fails its check, the load balancer stops routing to it. Your users see no interruption.

3. Caching

Caching is the single highest-leverage optimization in most systems. Before optimizing your database queries, ask whether the data needs to hit the database at all.

**Cache layers:**

| Layer | Location | Scope | Latency | |---|---|---|---| | Browser cache | Client | Single user | ~0ms | | CDN | Edge nodes | All users | ~5ms | | Application cache (Redis) | Server | All users | ~1ms | | Database query cache | DB engine | All users | ~5ms |

**Cache invalidation strategies:**

**TTL (Time to Live)** — expire after N seconds. Simple. Tolerates stale data.
**Write-through** — update cache when you update the database. Fresh. Write overhead.
**Cache-aside (lazy loading)** — read from DB on cache miss, populate cache. Simple. First request is slow.

**The hard part:** knowing what to cache and for how long. Highly dynamic data (user sessions, live inventory) needs short TTLs. Rarely-changing data (product catalog, static content) can cache for hours or days.

4. Database Partitioning

When a single database becomes a bottleneck:

**Vertical partitioning** — split tables by column. Store frequently accessed columns (name, email) in a hot table; rarely accessed columns (profile bio, preferences) in a cold table.

**Horizontal partitioning (sharding)** — split rows across multiple databases. User IDs 1–1M on shard A, 1M–2M on shard B.


# Shard key selection matters enormously
# Good: userId (evenly distributed)
# Bad: country (US has 10x more users than others — hot shard problem)

Sharding introduces complexity: cross-shard queries, resharding when you outgrow your split, transactional guarantees across shards. Delay sharding as long as possible. It's not a first-year problem for most companies.

5. Message Queues

Queues decouple producers (services that generate work) from consumers (services that do the work).


Order Service → [Order Queue] → Inventory Service
                             → Email Service
                             → Analytics Service

Benefits: - **Resilience** — if Email Service is down, orders don't fail; emails queue up and process when the service recovers - **Load smoothing** — absorb traffic spikes without overwhelming downstream services - **Retry handling** — failed messages retry automatically with configurable backoff

Common tools: RabbitMQ, Apache Kafka, AWS SQS.

6. CAP Theorem

Every distributed system makes a trade-off between:

**Consistency** — every read receives the most recent write
**Availability** — every request receives a response
**Partition tolerance** — the system continues operating despite network partitions

Network partitions happen — you can't opt out of P. So the real choice is: when a partition occurs, do you want **consistency** or **availability**?

Bank transaction: choose **Consistency** (stale balances are dangerous)
Social media feed: choose **Availability** (seeing a post 500ms late is fine)

7. The Architecture Decision Record

Document every significant design decision:

markdown
# ADR-012: Use Redis for Session Storage

Status: Accepted

Context User sessions are currently stored in database. At 50k concurrent users, session queries account for 40% of DB load.

Decision Move sessions to Redis with a 24h TTL.

Consequences + Reduces DB load significantly + Sub-millisecond session reads - Adds Redis as an infrastructure dependency - Sessions are lost on Redis failure (mitigated by clustering) ```

Decisions without documentation get re-litigated constantly. ADRs give future team members (and future you) the context for why the system is the way it is.

Where to Go Next

These fundamentals unlock every deeper discussion about system design: consistent hashing, two-phase commit, event sourcing, CQRS, service mesh, distributed tracing. Master these foundations first — everything else builds on them.