How to Route Between LLMs to Save Money (2026)

You route between LLMs to save money by routing the cheaper model only on the prompts where you have proven it matches or beats the model you run today, on your own traffic, and leaving everything else exactly where it is. That is the whole answer, and the part everyone skips is the word "proven", because a router does not prove anything. It hands you a menu of cheaper models and picks one on price, latency, uptime, or a general benchmark, before it has even seen the answer it is about to return.

So the honest version of the question is not "how do I route to a cheaper model", that part takes five minutes. It is "how do I know the cheaper model was actually good enough for this prompt", and that is a proof problem, not a routing problem. This piece is about the difference, why the routing half is the easy half, and how to get the saving without trading it for an invisible quality drop your customers find before you do.

How do you route between LLMs to save money?

Prove first, route second. Concretely: take the high-volume prompts that are actually burning your bill, and for each one test a cheaper model against the model you run now, on your own real inputs, until you can see whether the cheaper one holds. Route the ones that pass, leave the ones that fail on the expensive model, and keep the expensive model armed as an instant fallback for anything that drifts later. The saving comes from the prompts that pass, and on those, the saving typically lands in the 30 to 60% range. It is a band and not a hero number because it genuinely varies prompt by prompt, which is the whole reason you prove each one rather than swinging your entire bill at a single cheaper model.

The reason this order matters is that routing to the cheaper model is trivial and proving it held is not. Reaching a cheaper model is a base-URL change. Knowing it did not quietly mangle one field in your extraction schema, on the specific messy prompt you run ten thousand times a day, is weeks of measurement that no router does for you. If you invert the order, route first and check later, you find out from a customer, not a dashboard.

What does a router actually route on?

Infrastructure, mostly, and occasionally a general model of "which model suits this prompt". Neither is a check on the answer.

Take the most popular one honestly, because it is a genuinely good product. OpenRouter is a unified gateway: one OpenAI-compatible endpoint reaching 400-plus models, provider token prices passed straight through at a flat credit fee, with pooled uptime and fallbacks across providers (OpenRouter pricing). That is real value. If what you need is breadth and fast exploration across the whole model landscape, it is one of the best tools going, and I want to give it full credit before I draw the line.

Here is the line. Its routing sorts and fails over on price, throughput, latency and uptime, and even its smarter Auto mode routes your prompt to a model with a meta-model, before generation (OpenRouter Auto Router). Every one of those signals is about the request or the provider, not the response. Failover handles a provider going down; it does nothing about a provider that returns a confident wrong answer with an HTTP 200, because as far as the gateway is concerned that request succeeded. There is no per-request check that the cheaper model's output was actually good enough for your task. That is not a flaw, it is just a gap the platform leaves to you, the same way a payment rail does not audit what you bought.

The learned routers are cleverer but land in the same place. RouteLLM, the open-source framework from LMSYS, trains a router on preference data to send easy queries to a cheap model and hard ones to a strong model, and reports impressive numbers: up to an 85% cost reduction while maintaining around 95% of GPT-4 quality on MT-Bench (Ong et al., 2024; LMSYS, 2024). That is real research and worth reading. But notice what the router decides on: a learned judgement of the query, before the answer exists, calibrated against a public benchmark. It is a very good guess. It is still a guess about your traffic.

Why a menu is not a proof

Because the number that makes a router look like a money-saver is measured on the wrong thing. MT-Bench is not your support macros, your call summaries or your extraction schema, and public benchmarks are exactly the surface that gets contaminated and gamed, with leaderboard positions shifting on undisclosed testing and selective reporting rather than genuine capability (Singh et al., The Leaderboard Illusion, 2025). A model that routes well on a benchmark tells you almost nothing about whether it holds on the specific, slightly messy prompt you run ninety times a day.

And the routing decision happens at the wrong moment. A router picks a model before it has seen the answer, so it can only ever act on a prediction of difficulty, never on the quality of what actually came back. That is the structural gap: the one moment where you could check whether the cheaper model was good enough, the moment after it answered, is the one moment the router has already moved past. So "route to save money" quietly becomes "route on a guess and hope the benchmark was right about my traffic", which is how a visible cost problem gets traded for an invisible quality one.

This is also why the per-token price falling does not rescue you. Prices genuinely drop fast, and unevenly across tasks (Epoch AI), and inside a single vendor's line-up the gap between the frontier tier and the cheap tier is large: Anthropic's Opus and Haiku output tokens differ by roughly 5x (Anthropic pricing), with OpenAI's frontier-versus-cheap spread in the same ballpark (OpenAI pricing). The invitation to route down a tier is real and standing. Accepting it without proof is the mistake, and a router does not fix that, it just makes the mistake faster to make. For the mechanics of how routing decisions actually get made under the hood, I wrote up how AI model routing works separately.

Router versus proof layer: an honest side-by-side

Both are legitimate tools. They answer different questions, and I will concede plainly where a router wins, because the comparison only means anything if it is fair.

Read the "router wins" rows as real. If you want a 300-model menu, multi-provider failover, or to explore fast and cheaply, a router is the correct tool and I would point you at the OpenRouter comparison rather than pretend otherwise. What a router does not do, and does not claim to, is close the loop on quality. So if your only reason to route is to spend less, a router does the five-minute half and hands you back the hard half, and the hard half is the one that decides whether you saved money or just moved the problem somewhere you cannot see it.

So how do you actually get the saving?

You replace the routing decision for the cost job with a proof. Not "keep the router and bolt on an eval you have to build and maintain", but a layer that does the whole cost job end to end: reaches the cheaper model, proves it on your traffic, routes only what passed, and reverts the instant it drifts. That is the job a router leaves half-done, and it is the job worth doing properly.

The proof runs on three axes at once, because "good enough" means different things for different parts of an answer. Format is a hard exact-match gate: valid JSON, required fields present, correct shape, because a malformed response is the failure that returns HTTP 200 and stays invisible until a customer hits it, and format is not a matter of opinion so it is not left to a judge. Categorical covers the cases where the two models land on different answers, and those get re-judged blind, so a difference only counts against the cheaper model if a blind judge of your baseline's own class actually prefers the baseline. Semantic rides along as a diagnostic before any switch, informing the picture without being the gate that flips traffic.

The piece that ties it together, and the piece routers structurally cannot have, is the bar. You cannot call a swap worse until you know how much your own model already disagrees with itself, because ask your expensive model the same prompt twice and the answer changes. So you measure your baseline's own run-to-run consistency first, and a cheaper model passes when it disagrees with your baseline no more often than your baseline disagrees with itself. The standard comes from your model, not from a vendor and not from a leaderboard. The judge is never one of the contestants, because models favour their own family's output, a bias documented alongside position and verbosity bias in the foundational LLM-as-judge work (Zheng et al., 2023), and the judge reasons at your baseline's class, not below it. The full walk-through of that measurement loop, blind paired judging, swapped answer order, confidence intervals and all, is in how to prove a cheaper model is good enough.

There are two ways to start, and neither asks you to trust a claim. The offline path is the one I would use first, because it changes nothing in production: export a JSONL of past requests, upload it, and get the proof on your own historical prompts before you touch a line of code. The live path is a two-line SDK swap, after which every request still goes to your baseline provider exactly as before, the cheaper candidate runs in parallel on the real prompts, and only once it clears the bar does any traffic route to it, with instant fallback the moment output drifts off-shape. The honest headline is 30 to 60% lower cost with quality that matches or beats your current model, proven on your prompts, never "the same" and never "guaranteed".

One caveat stated plainly, because it is true: this is not for coding agents. Long-horizon agentic coding is exactly the broad, high-variance work where cheaper models still lose and no honest bar will tell you otherwise. Parity is for the high-frequency, well-defined jobs a business runs all day, the classification, extraction, summarisation, qualification and generation off structured data, and that is also where most of a bill is hiding. For the triage of which of your prompts even belong in this conversation, and which genuinely need the expensive model, see which prompts actually need the expensive model, and for the cost-per-task framing behind the cheap-model shortlist, the cheapest LLM APIs ranked by task not token.