> ## Content Index
> Fetch the complete content index at: https://blog.avis.net/llms.txt
> Use this file to discover other available public pages before exploring further.

# Build Your Own LLM Router or Use a Managed One? How to Decide
- URL: https://blog.avis.net/build-vs-buy-llm-router/
- Published: 2026-10-16T04:42:30.000Z
- Updated: 2026-10-16T04:42:44.000Z
- Description: An LLM router is easy to start and expensive to own. What building, self-hosting and buying really cost, and five questions to decide.
- Author: AVIS
- Tags: Blog, English

Every app that calls more than one model provider ends up with an LLM router, whether anyone planned one or not. It starts as a 200-line wrapper that picks a provider and retries on errors. Six months later it's translating request formats, tracking costs, rotating keys and paging someone at night. Multi-model setups are now the norm: in 2026 telemetry from thousands of companies, 69% used three or more models. This guide lists what a production router has to do, compares building, self-hosting open source and buying a managed gateway on what each really costs, and ends with five questions that make the choice clearer.

## What an LLM router has to do in production

An LLM router sits between your app and the model providers and does everything your app shouldn't have to know about: which provider to call, in which format, with which key, and what to do when the call fails. The first version only needs to forward requests. The production version needs the jobs below, and each one is either code you write, software you run, or something you pay someone else to run.

| Job                                                     | Build in-house                       | Self-host an open-source proxy     | Managed gateway                  |
| ------------------------------------------------------- | ------------------------------------ | ---------------------------------- | -------------------------------- |
| Translate between provider request and response formats | You write and update it              | Included; you upgrade it           | Included                         |
| Retries and fallbacks                                   | You write it                         | You configure it                   | Varies; check what's automatic   |
| Streaming through the router                            | You write it                         | Included                           | Included                         |
| API keys for every provider                             | You store and rotate them            | You store them in the proxy        | One key to the gateway           |
| Spend limits per key, team or app                       | You write it                         | You configure it; needs a database | Usually included                 |
| Cost tracking per request                               | You write it and keep prices current | Included; you keep prices current  | Included                         |
| Model catalog and deprecations                          | You track every provider             | You update config                  | The gateway updates it           |
| Database, cache and scaling                             | You run them                         | You run them                       | Not your problem                 |
| Monitoring, alerting and on-call                        | You                                  | You                                | Shared; you still watch your app |
| Security patches and dependency updates                 | You                                  | You                                | The gateway                      |

The list grows with every provider you add, because each one has its own formats, errors and limits. If you've read why [every provider's API is different](https://blog.avis.net/llm-api-differences/), you've seen how much of the translation job is edge cases rather than core logic.

## What building or self-hosting really costs

Building or self-hosting is cheap in licences and infrastructure, and expensive in engineering time. The software can be free. Running it reliably isn't.

### The infrastructure

A popular open-source proxy's own production checklist gives a good picture of what "self-hosted" means in practice: a Postgres database for keys and spend records, Redis as soon as you run more than one instance, at least 1 vCPU and 4 GiB of memory per instance (described as a floor, not a target), pinned versions instead of "latest", database migrations on every upgrade, and alerting for exceptions, budget crossings and database errors. None of that is unusual, and all of it is yours to run.

One published build-or-buy estimate from June 2026 puts the infrastructure alone at roughly $160 to $550 a month. Once it adds a quarter of a DevOps engineer's time and on-call cover, the total comes to about $3,200 to $4,500 a month, or roughly $38,000 to $55,000 in the first year, before you pay for a single model call. Your numbers will differ with salaries and traffic, but the ratio is the point: infrastructure is the small part.

### The engineering time

A custom router has the same running costs plus the cost of writing every feature in the table yourself. The first version is quick. The cost arrives later, as maintenance:

- **New providers and models.** Each one brings a new request format, new error codes and new limits to map.
- **Price changes.** Cost tracking is only as accurate as the price table behind it, and providers change prices without asking you.
- **Deprecations.** When a provider retires a model, every route that names it has to change. Our guide to [model deprecation](https://blog.avis.net/ai-model-deprecation-what-to-do-when-your-model-is-retired/) covers what that involves.
- **Security.** You own patching, dependency updates and key storage. In March 2026, a widely used open-source LLM proxy had two malicious releases published to its package registry, carrying code that stole credentials. Teams pinned to an earlier version weren't affected. Anyone who installed the bad releases had to treat every credential on those machines as exposed.

### The parts that are harder than they look

Fallback is the best example. "If the call fails, try another model" sounds like five lines of code. In practice it's at least six decisions: which errors should trigger it, how many retries come first, how long to stop sending traffic to an unhealthy provider, whether a context-length error should fall back to a model with a bigger window, whether the backup model is allowed for this user's budget, and how to test it without waiting for a real outage.

These decisions matter because failures are routine. In 2026 production telemetry, around 5% of model requests failed, and nearly 60% of those failures came from capacity limits. The same report pointed at excessive retries and inefficient routing as growing causes. A router that retries badly can make an outage worse and your bill bigger, a cost we cover in [the hidden costs of running AI in production](https://blog.avis.net/hidden-costs-ai-production/).

## What a managed gateway costs, and what it leaves to you

A managed gateway replaces most of those fixed running costs with a fee on usage. You give up some control in exchange for not running the database, the cache, the upgrades or the on-call rota.

### The fee

Managed gateways charge in different ways: a percentage of model spend, a markup on token prices, a monthly plan or a mix of these. AVIS charges the model's own price passed through unchanged, plus a 10% platform fee on API usage, with no setup fees, minimums or seats. See our [pricing guide](https://docs.avis.net/guide/get-started/api-pricing?ref=blog.avis.net) for the details.

The useful comparison is that fee against what self-hosting would cost you, not against a free licence:

| Monthly model spend | 10% platform fee | Self-hosted estimate (from above) |
| ------------------- | ---------------- | --------------------------------- |
| $1,000              | $100             | \~$3,200–$4,500                   |
| $10,000             | $1,000           | \~$3,200–$4,500                   |
| $40,000             | $4,000           | \~$3,200–$4,500                   |

On those numbers, a percentage fee stays cheaper until model spend reaches roughly $32,000 to $45,000 a month. Past that point, self-hosting starts to pay for itself, if you have the team to run it. Treat the self-hosted column as one estimate, not a benchmark, and replace it with your own salaries and traffic.

### The trade-offs

A managed gateway isn't free of costs beyond the fee:

- **An extra network hop.** Measure it from your region with your real payloads. For most generation workloads it's small next to model latency, but test before you assume.
- **Less control.** You can't change how the gateway routes requests internally or patch it yourself.
- **Data path.** Your prompts pass through a third party. Check the provider's data and retention terms against your compliance needs.
- **Lock-in.** Lower if the gateway speaks a standard API format. AVIS exposes [OpenAI-, Anthropic- and Gemini-compatible surfaces](https://docs.avis.net/api-reference/introduction?ref=blog.avis.net), so moving on or off is mostly a base URL and key change.

### The code you still keep

A managed gateway doesn't remove every line of routing code, and it's worth knowing what stays. On AVIS, errors from the model provider come back to you [unmodified](https://docs.avis.net/api-reference/introduction/errors?ref=blog.avis.net), with upstream outages returned as `502` or `503`, and the docs don't describe automatic retries across providers. So keep a short wrapper in your app that retries transient errors with backoff and switches to a backup model when the primary fails. That's a few dozen lines, not a service to run.

What the gateway does cover is the heavy part: one API key for 300+ models, one request shape across providers, [spend limits on every key](https://docs.avis.net/guide/get-started/avis-api-key?ref=blog.avis.net), and a [usage record](https://docs.avis.net/api-reference/introduction/use?ref=blog.avis.net) for every request with its model, status and final cost.

## How to decide: five questions

The right choice depends less on the tool than on how central the router is to your product and who'll run it. Answer these five questions honestly:

1. **Is the router itself part of what you sell?** If routing logic is a competitive advantage, building it can make sense.
2. **Do you already have a platform team with spare time?** Self-hosting needs people who'll upgrade, patch and answer alerts.
3. **Do data residency or compliance rules require the traffic to stay in your infrastructure?** If so, a managed gateway may be ruled out before cost comes into it.
4. **Is your model spend high enough that a percentage fee costs more than a team?** Use your own version of the break-even table above.
5. **Do several teams need separate keys, budgets and cost reports?** That's a lot to build, and most managed gateways include it.

| Your situation                                                  | Likely fit                    |
| --------------------------------------------------------------- | ----------------------------- |
| One app, small team, spend under a few thousand dollars a month | Managed gateway               |
| Several teams, no platform engineers to spare                   | Managed gateway               |
| Platform team in place, high volume, strict data rules          | Self-hosted open-source proxy |
| Routing is your product, or you need behaviour no tool offers   | Build in-house                |

You don't have to decide forever. A common path is to start managed, keep your code pointed at a standard OpenAI-compatible base URL, and revisit the choice when spend or compliance changes. If both the managed gateway and a later self-hosted proxy speak the same format, switching is a configuration change rather than a rewrite.

## FAQ

### What is an LLM router?

An LLM router is the layer between your app and the model providers. It decides which provider and model handle each request, translates request formats, applies retries and fallbacks, and records usage and cost. It can be code inside your app, a proxy you run yourself, or a managed gateway.

### Is it cheaper to build or buy an AI gateway?

For most teams, buying is cheaper until model spend is high. One published 2026 estimate puts self-hosting at roughly $3,200 to $4,500 a month once engineering time and on-call are counted. At a 10% platform fee, that equals about $32,000 to $45,000 a month of model spend.

### Is a self-hosted LLM proxy free?

The licence can be free, but running it isn't. You need a database, a cache, monitoring, upgrades, security patches and someone on call. In one published estimate, the all-in cost is roughly ten times the infrastructure bill.

### Does a managed gateway remove the need for retry logic?

Not always. Some gateways retry or fail over automatically, and others pass provider errors straight back to your app. Check the gateway's error documentation, and keep a short retry-with-backoff and fallback wrapper in your code either way.

### Can I switch from a managed gateway to a self-hosted router later?

Yes, if your code talks to a standard API format. When both options are OpenAI-compatible, switching is mostly a change of base URL and API key. Keep provider-specific fields out of your core code to make the move easier.

## Wrapping up

An LLM router is easy to start and expensive to own. Building makes sense when routing is your product, self-hosting makes sense when you have the team and the volume, and a managed gateway makes sense for most teams until spend or compliance says otherwise. Whichever you pick, keep a small retry and fallback wrapper in your app and a standard API format underneath it. If you want to try the managed route, AVIS gives you one API key for 300+ models through [OpenAI-, Anthropic- and Gemini-compatible endpoints](https://docs.avis.net/api-reference/introduction?ref=blog.avis.net), at the model's own price plus a 10% platform fee.

*Last updated: October 2026*