LLM gateways & routers

What it is

An LLM gateway sits between your app and model providers. It gives you one API and key for hundreds of models, retries and falls back when a provider is down, caches responses, enforces budgets and rate limits, and logs usage. Some (aggregators) resell model access; others only proxy your own provider keys.

How pricing works

Three models exist: a percentage fee on credits you buy (aggregators), a flat subscription with included request logs plus overage, or free open-source self-hosting. We normalise to a month with 1M requests and about $5,000 of underlying model spend; model tokens themselves are billed at the providers' list price and are not included.

We compare everything in one unit: USD per month at 1M routed requests (≈ $5,000 of model spend).

Watch out for

• A percentage fee grows with your spend; compare it to a flat plan once usage is large.
• Gateways add a network hop; check added latency and whether the gateway itself becomes a single point of failure.
• Data passes through a third party; review retention and compliance.