Stateful or Stateless? It's Really About Where the State Lives

“Make the service stateless so it scales” is one of the most repeated pieces of architecture advice, and one of the most misunderstood. Stateless doesn’t mean the state disappeared. State almost never disappears — a login has to be remembered, a shopping cart has to persist, a session has to know who you are. “Stateless” just means the state isn’t in the service instance anymore. It moved. And the actual design work isn’t eliminating state; it’s deciding where it lives and paying attention to what that choice costs.

Why “stateless” scales, and what it really means

The reason stateless service instances scale so well is simple: if an instance holds no session-specific state, then any instance can handle any request, you can run a dozen behind a load balancer, and you can add or kill them freely. One falls over? Its requests just go to another; no user notices, because that instance wasn’t holding anything unique. That’s genuinely powerful and it’s why the advice exists.

But notice what had to happen for that to work: the state the instance used to hold had to go somewhere the other instances can also reach. Statelessness at the service tier is bought by moving state out of the service tier. The scaling win is real; it’s just not free, and pretending the state vanished is how people get confused when it resurfaces as a bottleneck somewhere else.

Stateless is a lie you tell about the service, not about the system. The state is still there — you just moved it to where you could scale around it.

The three places state can actually live

When you push state out of the service instance, it has three honest destinations:

• In the client. Hand the state to the client and have it present it on each request — a signed token carrying who they are, a cart the client holds. The server keeps nothing between requests. Great for scaling (the server truly holds nothing), but the client now carries state you have to trust, which means signing it so it can’t be tampered with, and living with the fact that you can’t easily revoke or change it mid-life because it’s out there in the client’s hands.
• In a shared store. Put the state in something all instances can reach — a cache like Redis, a replicated in-memory store, a database. Instances stay stateless; the store holds the truth. This is the common answer, and it just relocates the hard questions: now the store is the thing that must scale and stay available, and you have to decide whether it’s allowed to lose that data or not.
• Recomputed on demand. Sometimes the cheapest “storage” is none — derive the state fresh each time from a source of truth rather than holding it. Trades compute for storage, and only works when recomputation is cheap enough.

Every “stateless” architecture is really a choice among these three for each piece of state — and often a different choice per piece.

The specialized version: a replicated state store

There’s a pattern worth knowing for state that’s too hot for a database and too important to lose on one node: a replicated in-memory store, where each node keeps state in RAM and replicates it to peers, so any node can answer and no single node’s death loses it. It’s how systems track things like “how many active sessions does this identity have right now” at high speed across a cluster — state that’s shared, fast, and deliberately not persisted to disk because its whole value is being in-memory and current. It’s a great illustration that “where does the state live” has richer answers than “client or database”: sometimes it’s replicated across the very tier you called stateless, which is a different tradeoff again (fast and resilient, but bounded by memory and gone on a full outage).

The cost you’re actually choosing

Line the options up and you’re really choosing which problem you’d rather have:

• Client-held state → best scaling, but trust, size, and revocation problems (you can’t un-issue a token easily).
• Shared store → simple mental model, but the store becomes a critical dependency you must scale and protect, and you inherit its consistency and lifetime questions.
• Recomputed → nothing to store or leak, but you pay in compute every time, and only if the source is fast.
• Replicated in-memory → fast and node-failure-resilient, but memory-bounded and not durable.

There’s no free option — there’s a choice about where the difficulty goes. And a subtle trap: if the shared store is your answer, remember that the client’s routing decides which node gets hammered, so “stateless and load-balanced” can still concentrate on one poor backend.

Ask “where does the state live?” not “is it stateful?”

The framing I’ve landed on: “stateful vs stateless” is the wrong question, because the answer is always “there’s state; it lives somewhere.” The useful question is where does each piece of state live, and what does that location cost me — in scaling, in durability, in trust, in complexity. Answer that per piece of state and you’ve actually designed the system, instead of chanting “stateless” and being surprised when the state you moved shows up as the new bottleneck. State is conserved. Design is deciding where to keep it. If you’ve moved state around to fix a scaling problem and learned where the cost really landed, I’d like to hear about it.