Failover — workshop

Route one stream of traffic across a bank of provider pipes that go dark without warning — failover is only instant if a backup was already warm, and warm capacity bills every tick whether or not it's carrying anything.

Four provider pipes, one stream of traffic. Exactly one pipe is active at a time; the rest sit cold (free, unprovisioned) unless you warm them. The active pipe goes dark at random, with no telegraph — a partner cutting you off mid-run. If a pipe was already warm, failover is instant. If nothing was warm, traffic drops to zero until a cold pipe finishes cold-starting, and that gap is gone the moment it happens — there’s no catching up on dropped requests. The only control is which pipes to keep warm, toggled per pipe with 1–4 or a click. Warm capacity costs the same whether or not it’s serving, so leaving everything warm “just in case” bleeds margin as reliably as an uncovered outage does — the skill is buying exactly as much insurance as the next outage is worth.

Riffs on OpenAI pulling its models out of Cursor on twelve weeks’ notice, and the LLM-gateway framing of surviving a cutoff like that as a config change rather than an outage — which only holds if a fallback provider was already warm.

Reuse

The mechanic lives in src/failover.js as a framework-free ES module:

createFailoverSim(opts) — returns a state object; call update(dt) each frame and toggleWarm(i) to cycle a pipe between cold, warming, and warm (a no-op on the active pipe or a dark one).
opts: pipes (array of { name, price, rate }), incomingRate, warmupTime, outageDuration, cutoffMin/cutoffMax, valuePerUnit, duration (0 for an endless run).
State exposes pipes (each with status — 'cold' | 'warming' | 'warm' | 'active' | 'dark' — plus rampTimer, darkTimer, cutoffTimer), totalOffered, totalServed, totalDropped, totalSpend, lastEvent / lastEventPipe (which pipe an outage/failover/cold-start just happened to, for UI flashes), and status ('playing' | 'finished'). No rendering or input in the module itself — the demo’s canvas drawing and click/key handling are reference code, not part of the reusable piece.

Gotchas

A pipe reaching the end of its warm-up lands as active if the stream is currently uncovered, or as warm standby otherwise — the same countdown drives both a cold start under fire and routine pre-warming. That’s deliberate: it’s what lets a mid-outage cold-start finish straight into serving instead of parking as a useless standby nobody’s routing to.
Default pipe rates aren’t all wide enough for incomingRate (5/s): Cinder (4/s) and Local (3/s) drop the overflow even while active and perfectly healthy. Reliability and cost aren’t the same axis — the cheapest pipe is also the leakiest one.
The demo bundles its own copy of the module (self-contained by contract). If you touch src/, re-copy it into demo/.