The Cut You Stop Paying on Every AI Request

If your team already runs AI inside a few workflows, an enrichment step here, a summarizer there, a drafting assistant your reps lean on, you are almost certainly paying a routing vendor a percentage on top of the models themselves. Load a thousand dollars of credits into OpenRouter and about nine hundred forty-five of it reaches the actual inference; the rest is the 5.5% fee it charges to top up your balance, with a floor that makes small purchases hurt more. At ten thousand a month that cut is five hundred fifty dollars a month, every month, for the service of pointing your requests at the right provider. OmniRoute is the self-hosted version of that middleman: one endpoint in front of 290-plus providers, released under an MIT license, running on a machine you already own. The fee goes to zero. What the fee was quietly buying you does not, and that is the part worth reading past the headline for.

What the repo actually is

OmniRoute is a gateway. You run it, it exposes an OpenAI-compatible API at a local address, and anything that already speaks the OpenAI format, Claude Code, Cursor, Cline, Copilot, or your own scripts, points at that address instead of at a single provider. Behind the one endpoint it can reach more than five hundred models across close to three hundred providers, ninety-odd of which have free tiers. When one provider throttles you or falls over, it fails over to the next one that can serve the request. It keeps your API keys on your own machine rather than handing them to a hosted service. And it ships a token-compression layer the maintainer claims trims anywhere from fifteen to ninety-five percent off a request depending on what you send it.

The traction is real and recent. The project cleared roughly eighteen thousand stars during July, which puts it among the faster-climbing repositories on GitHub this month, and it is actively released rather than parked, with contributions credited to a few hundred developers. That matters because star count on its own is noise; a repository adding thousands of stars a week while shipping releases is one people are actually running, not just bookmarking.

The install story is genuinely two stories, and the honest answer to "could a non-engineer stand this up" depends on which one you pick. There is a Docker path and a Node path, which is the normal semi-technical route: if you have ever run a container, you can have this going in an hour or two. And there is a desktop app, packaged for you, which is close to one-click and is the version a RevOps manager who has never touched a terminal could plausibly get running on a Friday. Say which one you are using out loud, because the desktop app quietly changes who this repo is for.

Why it matters to a small operator

There are two readers who should care about this one, and they care for slightly different reasons.

The first is the operations or RevOps person who has already wired AI into a handful of processes and is now the accidental owner of an AI bill nobody is watching. You know the shape of it: three or four tools, each with its own key, each billing separately, and no single place to see what the whole thing costs or to cap it. The tidy internal answer is a gateway that consolidates the keys, routes intelligently, adds a spend limit, and fails over when a provider has a bad afternoon. That is a real project, and it is exactly the kind of project that sits in an engineering backlog for a quarter because it is nobody's priority and everyone's someday. OmniRoute is that project, already built, that you can run yourself instead of waiting for it to get prioritized. The displaced cost here is not really the 5.5%. It is the six-week ticket that never clears.

The second reader is the owner-operator watching the number. If your monthly AI spend is small, the percentage is small too, and the desktop app plus a free-tier provider or two can genuinely take your out-of-pocket toward zero for low-volume work like drafting, summarizing, or answering routine questions. For a two-thousand-a-month spend the fee you are skipping is a hundred and ten dollars; not life-changing, but not nothing, and the failover means a single provider outage stops taking your tools down with it.

For both readers the pitch is the same underneath: you stop renting the endpoint, and you stop being locked to one model vendor's pricing and one vendor's uptime. When a cheaper or better model shows up, it is a config line, not a migration.

The honest take

Here is where the free number stops doing all the work.

Start with who maintains it. OmniRoute has a few hundred contributors on paper but one primary author, and that is the single most important fact about putting it in a path your business depends on. If that person loses interest, the repository does not vanish, MIT means you can fork it and keep going, but you inherit an unmaintained proxy sitting between your team and every AI tool it uses. The mitigant is that a gateway is swappable by design; it speaks a standard format, so ripping it out and pointing back at a provider directly is a bad afternoon, not a rebuild. Still, plan for the maintainer walking away, because with a solo-led project you are underwriting that risk whether you think about it or not.

Then the part the "ninety free providers" line is quietly hiding. Free inference is free for a reason, and the reason is often that your prompts are the product. Some free endpoints rate-limit hard, some deprecate without warning, and some reserve the right to log or train on what you send them. Routing an internal brainstorm through an unknown free provider to save a few tokens is fine. Routing anything with a customer's name, a contract, or personal data through that same provider to save the same few tokens is a data-governance problem you created to shave a bill that was already small. If you run this, pin anything sensitive to providers you actually trust and treat the free pool as strictly for low-stakes work. The tool gives you the switch; it does not make the judgment for you.

The compression deserve the same skepticism. Trimming fifteen to ninety-five percent off a request is a wide range, and the way you get the high end is by rewriting or dropping context. On a summarization task, where you want the gist anyway, that is close to free money. On an exact-match task, extracting fields from an invoice, reviewing contract language, generating code that has to compile, lossy compression can change the answer in ways you will not catch until it is in production. Test compression on your actual workload before you trust it on the tasks where being slightly wrong is expensive, and be willing to turn it off for those.

And then the cost the fee was actually covering. OpenRouter's 5.5% was never really the price of routing, which is cheap. It was the price of not being the person on call when a provider changes its API at midnight, not maintaining the failover logic, not rotating keys, not keeping the thing patched. Self-hosting moves all of that onto you. The dollar cost of hosting is trivial, a small VPS runs six to twelve dollars a month, or you run it on hardware you already have for nothing, but the real cost is that you now own uptime and upkeep for a component in your critical path. For a team with someone technical enough to babysit it, that trade is obviously worth it. For a team where the AI stack is already held together with hope, adding a self-hosted gateway you have to maintain may cost more attention than the fee ever cost in dollars.

One more, quietly: concentrating every provider key into one gateway is convenient and is also a single place where a breach hurts more. Keys on your own machine is better than keys in someone's cloud, right up until that machine is the one that gets popped. Lock it down like the credential store it now is.

What to do about it

The decision is not "free versus 5.5%," because stated that way the answer is too easy and wrong. The decision is whether your team has the fifteen or twenty minutes a month of attention that self-hosted infrastructure demands, and whether your AI spend is large enough that the percentage you are skipping pays for that attention. Below a couple thousand a month, run the desktop app for the low-stakes work, keep your sensitive workflows pointed straight at a provider you trust, and enjoy the failover. Above five figures a month, the fee is a real salary-sized line item and standing up OmniRoute on a box someone owns is straightforwardly worth the upkeep. In the messy middle, the tiebreaker is not money at all; it is whether there is a person who will notice when it breaks.

Whichever way you land, the thing this repo makes newly obvious is worth sitting with. The routing was always cheap. What you were paying for was someone else holding the pager, and the question OmniRoute actually puts in front of you is whether that pager is yours to hold.