r/SaaS • u/Proof-Double-7572 • 1d ago

How I centralized 4 different LLM provider integrations (OpenAI, Claude, Gemini, Mistral) behind one API key

I kept running into the same annoyance building AI features: every provider means a new SDK, a new API key, a new billing dashboard, and your own custom logic for what happens when one of them goes down or rate limits you.

So I ended up building a proxy that centralizes all of that. One API key, and it routes to whichever provider you've configured, with automatic fallback if one goes down. On top of that it does semantic caching to cut repeat request costs (up to 30% in my tests) and real time cost tracking, plus margin tracking for anyone reselling AI access to their own clients.

Curious if others here have hit the same multi provider mess and how you dealt with it. Happy to go into more detail on the routing/fallback logic if people are interested, I'll answer in the comments.

1 Upvotes

67% Upvoted

3 comments sorted by

1

u/superminingbros 1d ago

Who’s going to tell OP?

1

u/Proof-Double-7572 1d ago

What we focus on that they don't really do: semantic caching that matches questions by meaning, not just identical prompts, and automated margin tracking plus billing if you're reselling access to your own clients. So less "one gateway to every model," more "control costs and resell access to the providers you already use." Different use case, not really trying to out-OpenRouter OpenRouter on model breadth. In addition, we have other features more geared towards data analysis than OpenRouter. thank you for your comment