A live router, not a mock-up
Every request is classified in about half a second, every model is priced for this request at what it actually costs including the risk of having to redo it, and the cheapest expected total wins. Then the answer streams back from whichever model that turned out to be.
Waiting for the candidates…
| Model | Capabilityin this topic | Price / 1Min · out · cache read | P(success)measured | This turncall cost | Expectedincl. risk of a retry |
|---|
The part most routers get wrong
A provider that has already read your conversation bills the next turn at roughly a tenth of the input price. Move to another model and the whole prefix is written again at full price. So the question at every turn is not "which model is best" but "is the better model worth leaving a warm cache for" — and the answer changes as the clock runs out on the cache.
This is the live policy object answering a question instead of a request — the only honest way to show what a 100k-token agent prefix does to a decision without paying for one. Free tiers are hidden by default: a free route has no cache price, so with them in the catalog there is nothing to trade off.
| Route | Cache | This turn, staying | This turn, cold | The cache is worth | Expected total |
|---|
Only the clock changes between these rows. When the cache expires, staying stops being worth anything and the decision flips.
A real conversation with a pasted document, routed turn by turn by the same router. Every answer is really generated; the strip under each one is what the router decided and why. Use the pause control to make the clock jump before you send.
Measured, 17–18 September 2026
Eight models on 78 graded tasks, a replay of one week of real coding-agent traffic (1,638 sessions, 57,696 calls, 8.7 billion input tokens of which 96 % were cache reads), and a live run of the router itself. Read the dollar columns as arithmetic: real traffic and measured success rates priced at public list prices. No invoice was compared. No money was saved and none was measured.
Loading the numbers…
How it works
One call to Jev, TypeSafe's System One model, on a scrubbed and truncated summary of the turn — never the raw prompt. It answers typed questions with calibrated probabilities: topic, difficulty, whether tools or a long context are needed, whether this builds on the previous turn, and how costly a subtly wrong answer would be. About 0.65 s. On our 78 tasks it named the topic correctly every time, and its difficulty score predicted which tasks a cheap model would fail better than the task's own difficulty label did.
The catalog is not a hard-coded price list. Every model is looked up in the Benchmark Heaven API: capability per topic, list price per offer including the cache read and cache write price, context length, and a benchmaxxing penalty — each number carrying its basis and how strong that basis is. Cached locally with a TTL, so a benchmark site being down never takes routing down.
Capability read from benchmark headlines mis-ranks specific models and effort levels. Where we have measured a model on a topic and difficulty, the measurement wins. That single change was worth three points of task success and cut spend to a third in the replay.
For each candidate: what this turn costs there given what it still holds in cache, times the chance it gets the turn right, plus what a failure would cost — a retry on a stronger model if the failure is noticed, the price of a wrong answer if it is not. Lowest total wins. Inside an agent's tool loop the router never switches, because that throws away a cache that is seconds old.
The router is an OpenAI-compatible proxy. Point any tool that speaks
/v1/chat/completions at it and it routes per user turn:
git clone https://github.com/fstandhartinger/auto-model-router
cd auto-model-router
pip install -r requirements.txt
cp examples/config.example.yaml config.local.yaml # your endpoints, keys by env-var name
export AUTO_ROUTER_CONFIG=config.local.yaml
export TYPESAFE_API_KEY=... # the classifier
uvicorn auto_router.server:app --port 8795Then point a coding agent at it:
export OPENAI_BASE_URL=http://127.0.0.1:8795/v1
export OPENAI_API_KEY=anything
opencode run -m "auto/auto" # or any OpenAI-compatible clientA flat-rate coding subscription may only be used through its own official client — the router will price one as a tier if you configure it, but do not put a proxy in front of someone else's plan.
A router should not care whether a model runs in a data centre or in somebody's browser. The Bonsai swarm is a set of volunteers running a ternary 27B model on WebGPU, reachable over the same OpenAI-compatible shape as every other provider. It enters the catalog with a price of zero, a small context window, no prefix cache and a modest capability — and then competes on those numbers. It wins easy and medium turns and loses hard ones, which is exactly what an expected-cost rule should do with a cheap, slow, best-effort route.
Read live from the benchmark API when the demo started. The last column is the exact basis of that model's coding capability — hover it for every other topic.
| Model | Context | Price / 1M in · out · cache read | Cache TTL · hit rate | Capability basis |
|---|
Privacy
Your prompt is sent to two places: to TypeSafe for classification (scrubbed of recognisable credentials and truncated first), and to the model the router picks, which is hosted by a third-party inference provider. If the peer-to-peer route is chosen, your prompt is processed on a volunteer's computer and the page says so before you use it.
Nothing that you typed. The server keeps counters — how many runs an address has started this hour, how many runs and how many cents the demo has used today — and a prompt-free routing record in memory for the page you are looking at. Your address is only ever held as a salted hash that is thrown away when the process restarts, and the salt changes with every restart.
Multi-turn conversations are kept in the server's memory for the length of your session so the cache logic has something to be about. They are dropped when the session falls out of the ring buffer or the process restarts, and they are never written to disk.
This is a public playground running on third-party inference providers and, when it is online, on strangers' GPUs. Treat everything you type as public.
No cookies, no analytics, no third-party scripts. The page stores one session id and your theme choice in your browser's local storage.
productivity-boost.com Betriebs UG (haftungsbeschränkt) & Co. KG — see the Impressum.
Impressum
productivity-boost.com Betriebs UG (haftungsbeschränkt) & Co. KG
Vertreten durch die productivity-boost.com Verwaltungs UG (haftungsbeschränkt),
diese vertreten durch den Geschäftsführer Florian Standhartinger
Kirschbaumweg 4
94032 Passau
Deutschland
E-Mail: florian.standhartinger@gmail.com
Umsatzsteuer-Identifikationsnummer gemäß § 27a UStG: DE296812612
Florian Standhartinger, Anschrift wie oben.
Diese Seite ist eine kostenlose, nicht-kommerzielle technische Demonstration. Die Antworten stammen von Sprachmodellen Dritter und können falsch sein. Für Inhalte externer Links sind deren Betreiber verantwortlich.
Plattform der EU-Kommission zur Online-Streitbeilegung: ec.europa.eu/consumers/odr. Wir sind nicht verpflichtet und nicht bereit, an Streitbeilegungsverfahren vor einer Verbraucherschlichtungsstelle teilzunehmen.