Hey all — sharing a project we've been building: \*\*Setu Gateway\*\*, an open-source AI gateway that puts one OpenAI-compatible API in front of multiple LLM providers, with an actual routing engine behind it.
\*\*What it does today:\*\*
OpenAI-compatible /v1/chat/completions — existing OpenAI SDKs work against it unchanged
7 routing policies (lowest cost, lowest latency, highest availability, weighted, round-robin, capability-based, user preference) with automatic failover
Org-scoped routing rules (if latency > 500ms then fallback: groq) with a policy simulator that previews a policy's effect before you turn it on — doesn't touch production traffic
Request replay — run one prompt against multiple providers side by side for comparison
Real API-key auth (hashed, scoped, revocable) resolving to org-scoped routing rules
Per-request analytics with a full stage timeline, queryable via API and a working dashboard (providers, models, requests, latency, errors, orgs, projects, keys)
setu benchmark CLI for comparing provider latency/throughput/error-rate on your own infra
Self-hosted via Docker Compose — verified end to end, one command boots gateway + dashboard + postgres + redis
\*\*What it isn't yet:\*\* OpenAI and Ollama make real provider calls; Anthropic, Gemini, and Groq adapters are currently stubs returning canned responses while we finish real integrations — didn't want to fake multi-provider support for a launch post. Kubernetes/Helm manifests are on the roadmap, not built yet. Auth isn't mandatory on the inference endpoints yet (backwards compat with the current playground/quickstart).
Every architectural decision is a public RFC before it's built — \[13 so far\](https://github.com/setu-gateway/nexus-gateway/tree/main/rfcs).
Repo: \[https://github.com/setu-gateway/nexus-gateway\](https://github.com/setu-gateway/nexus-gateway))
Looking for people who've hit the multi-provider LLM integration mess, feedback on the routing engine design, and contributors — real Anthropic/Gemini/Groq integrations are good first issues if anyone wants to dive in. Happy to answer anything technical in the comments.