Open Source · 2026
Routiform
Self-hosted LLM gateway: one OpenAI-compatible endpoint in front of 59 providers, with retries, multi-account failover and cost-aware routing.
- Project Year
- 2026
- Public Links
- 2
The problem
An agent stops mid-task because one provider rate-limited the key. Every new provider means another client rewrite and another secret to store, and nothing in the stack can say what the run cost.
Solution approach
- Put 59 providers behind one OpenAI-compatible /v1 surface (chat, embeddings, images, audio, video, rerank, and search), translating each request and response to the provider's own format.
- Built retries, multi-account failover, model-family fallback, and an emergency fallback tier into the default request path so a dead provider degrades gracefully.
- Added cost-aware routing with weighted, priority, and power-of-two-choices strategies, budget and quota controls, and a dashboard for providers, keys, usage, logs, and endpoint health.
- Shipped an MCP server with 25 tools and A2A v0.3 support so agent frameworks reach the gateway natively.
The outcome
- Around 10,600 npm downloads in the 30 days to August 2026, from a package first published in April 2026.
- Each new provider is added as a registry entry, keeping 59 integrations maintainable by one person.
- Cursor, Codex, Claude Code, Cline, and MCP clients all point at one base URL, making provider changes a single config edit.
Tech stack
TypeScriptNode.jsOpenAI-compatible APIMCPDocker