Skip to content

Open Source · 2026

Routiform

Self-hosted LLM gateway: one OpenAI-compatible endpoint in front of 59 providers, with retries, multi-account failover and cost-aware routing.

Project Year
2026
Public Links
2
01

The problem

An agent stops mid-task because one provider rate-limited the key. Every new provider means another client rewrite and another secret to store, and nothing in the stack can say what the run cost.

02

Solution approach

  • Put 59 providers behind one OpenAI-compatible /v1 surface (chat, embeddings, images, audio, video, rerank, and search), translating each request and response to the provider's own format.
  • Built retries, multi-account failover, model-family fallback, and an emergency fallback tier into the default request path so a dead provider degrades gracefully.
  • Added cost-aware routing with weighted, priority, and power-of-two-choices strategies, budget and quota controls, and a dashboard for providers, keys, usage, logs, and endpoint health.
  • Shipped an MCP server with 25 tools and A2A v0.3 support so agent frameworks reach the gateway natively.
03

The outcome

  • Around 10,600 npm downloads in the 30 days to August 2026, from a package first published in April 2026.
  • Each new provider is added as a registry entry, keeping 59 integrations maintainable by one person.
  • Cursor, Codex, Claude Code, Cline, and MCP clients all point at one base URL, making provider changes a single config edit.
04

Tech stack

TypeScriptNode.jsOpenAI-compatible APIMCPDocker
05

Links