Designing a LanCache for Patch Day, Not Benchmark Day

A local game-content cache is valuable because it reshapes repeated WAN downloads into controlled LAN traffic, but storage, DNS behavior, cache misses, and concurrency determine whether it actually helps.

Shift WAN Peaks Into Predictable LAN Traffic — Only When You Hit The Cache

A LanCache’s job is simple on paper: serve the same bytes locally instead of re-downloading them. In practice the space between “request” and “served” contains CDN behavior, range requests, conditional GETs, and client-side validation. There are two distinct execution paths that decide whether you get the benefit:

• Cache hit path: DNS or routing points the client to the cache; the cache returns stored content (possibly from kernel page cache) without touching the origin. Fast reads, low WAN egress, small blast radius.
• Cache miss path: cache either forwards the client to origin or fetches content itself, then stores and serves. This creates WAN traffic, write amplification, and potential client delay.

Design acceptance criteria should be written as “X% of requests for artifact class Y must be satisfied from the cache during a patch window” rather than raw throughput numbers.

DNS Interception Is The Control Plane — And It’s Also Fragile

DNS is the practical control plane for many LanCache installs: change resolution so clients talk to the cache. That gives you an easy toggle and visibility, but it brings failure modes:

• Client bypass: clients using DoH/DoT, hardcoded DNS, or hosts entries avoid your interception and fetch from origin.
• Split-horizon mismatches: inconsistent TTLs or recursive resolver failures lead to partial coverage and double downloads.
• Negative caching and NXDOMAIN: misconfigured interception can return errors or stale answers and break clients.

Treat DNS interception like an adapter with a contract: it must be auditable, reversible, and have a clear rollback. Least privilege applies—limit which records you rewrite and keep a short TTL for testing. Always plan for a safe failure mode where the client falls back to origin in a controlled way rather than silently failing.

Storage Isn’t A Benchmarker — It’s A Contention Surface

Storage choices show up as real operational problems on patch day. Benchmarks quoting sequential MB/s are not the whole story.

Failure modes and tradeoffs to consider:

• Concurrency: many clients reading the same large file can thrash IOPS if the kernel page cache is cold or if reads are hitting slow disks. SSDs mask this but cost more and change rebuild characteristics.
• Write path: a miss forces a write of large objects. If writes compete with reads, the read tail latency rises; if the filesystem or object store uses heavyweight metadata operations, you get stalls.
• Partial-serving: some game clients use range requests. If your cache writes to a temp file then renames on completion, you may be unable to serve ranges while the file is incomplete.

Design constraints: choose a storage layout that supports single-writer, multi-reader semantics, avoid fsync-heavy operations on hot paths, and prefer streaming writes that allow tail readers. Decide whether losing cache contents during an outage is acceptable; that decision drives durability choices.

From Miss To Hit: Control The Cold-Start And The Thundering Herd

Cold-starts are the single largest operational surprise. On day zero a popular 50–100GB patch requested by many clients can create a “thundering herd”: many clients fetch concurrently, the cache either fetches multiple copies or is overwhelmed serving while writing.

Practical mitigations:

• Single-flight locks: ensure one fetch per artifact, queue or stall other requests until the first completes.
• Pre-warm popular artifacts or stage them during off-peak windows using a dry-run list.
• Rate-limit origin fetches and apply backpressure to clients (short, documented retry policies) so failures degrade gracefully.

Decide the acceptable blast radius for origin fetch failures: return an error to clients, redirect to origin, or serve partial content. Each choice has costs: user-facing errors, WAN egress, or corrupted installs.

Validation Checklist: Patch-Day Dry Run

This checklist is a reusable artifact to exercise the control plane, storage, and miss paths before a real patch window.

• DNS and Interception

  • Verify A/AAAA resolution for a target CDN hostname points to the cache from multiple client subnets.
  • Test client DoH/DoT bypass: simulate clients resolving via external resolvers.
  • Validate rollback: return original DNS answer and confirm no persistent client errors.

• Cache Hit/Miss Behavior

  • Cold-start: request a new large artifact from a single client and observe origin fetch, storage write, and subsequent hits.
  • Herd simulation: concurrently request the same artifact from many clients; confirm single-flight behavior and that only one origin fetch occurs.

• Storage and Concurrency

  • Disk stress: run concurrent large-file reads while writing new artifacts; monitor read latency and IOPS.
  • Partial-range correctness: verify range requests succeed during and after the write.

• Failure Modes

  • Simulate origin failure mid-fetch; check how clients react and whether the cache leaves partial files that block future fetches.
  • Induce DNS failure and confirm clients fall back in a controlled manner.

• Metrics & Acceptance

  • Capture: cache hits, cache misses, origin egress, disk IOPS, per-artifact lock time.
  • Acceptance: defined qualitative thresholds (e.g., majority of patch traffic served locally, origin egress bounded by policy).

Run these in a staging VLAN and measure before enabling production traffic. The checklist is a dry-run sequence, not an ad-hoc smoke test.

Tradeoffs, Costs, And When You’re Wrong

A LanCache reduces WAN egress and centralizes control, but it adds operational complexity and a new failure domain. Costs include storage footprint, ongoing maintenance, DNS configuration management, and handling of edge cases like encrypted resolution or vendor endpoint changes.

When it’s a poor fit:

• Fleets so small that duplicated downloads are rare.
• Clients that validate content against a signature that requires origin contact.
• Environments where clients use enforced DoH or pinned certs that prevent interception.

When the design is right: your acceptance criteria are met in dry runs, you can toggle DNS safely, and the storage subsystem is tuned for concurrent reads and streaming writes.

Takeaway

Design the cache around behavior, not benchmarks. Treat DNS interception as the control plane and storage as a contention surface. Validate hit/miss transitions, simulate the herd, and hold a rollback plan. If acceptance criteria and dry runs don’t confirm predictable LAN traffic during a patch window, you’re designing for a benchmark, not for the day that matters. /contact