Skip to content

Open Source · 2026

Routiform

Self-hosted LLM gateway: one OpenAI-compatible endpoint in front of 59 providers, with retries, multi-account failover and cost-aware routing.

Project Year
2026
Android Downloads
Not publicly disclosed
iOS Downloads
Not publicly disclosed
Public Links
2
01

The problem

An agent stops mid-task because one provider rate-limited the key. Every new provider means another client rewrite and another secret to store, and nothing in the stack can say what the run cost.

02

Solution approach

  • Put 59 providers behind one OpenAI-compatible /v1 surface — chat, embeddings, images, audio, video, rerank and search — translating each request and response to the provider's own format.
  • Made resilience the default path rather than a setting: retries, multi-account failover, model-family fallback and an emergency fallback tier, so a dead provider degrades instead of stopping the workflow.
  • Added cost-aware routing — weighted, priority and power-of-two-choices strategies with budget and quota controls — plus a dashboard for providers, keys, usage, logs and endpoint health.
  • Shipped an MCP server with 25 tools and A2A v0.3 support so agent frameworks reach the gateway natively.
03

The outcome

  • Around 10,600 npm downloads in the 30 days to August 2026, from a package first published in April 2026.
  • Adding a provider is a registry entry rather than an adapter, which is what keeps 59 of them maintainable by one person.
  • Cursor, Codex, Claude Code, Cline and MCP clients all point at one base URL, so switching provider is a config change instead of a client rewrite.
04

Tech stack

TypeScriptNode.jsOpenAI-compatible APIMCPDocker
05

Links