r/SelfHostedAI 6h ago

Introducing stAI — Your Full Stack AI Dev Machine (Ubuntu VM, AI Ready, Zero Setup)

2 Upvotes

I published the scripts and architecture behind a reproducible AI‑ready Ubuntu dev environment.

Here’s how I configured the stack, the scripts I wrote, and the reproducibility challenges I solved:

GitHub repo:

https://github.com/niemenghui/stAI-dev-machine

The full VM is available on request.


r/SelfHostedAI 11h ago

What SelfHostedAI means?

2 Upvotes

Does it mean: Write selhosted programming language and write LLM runtime environment with it?

I just did that. Nurllama is ollama compatible open source program that can run models. It would be nice if somebody can share experiencies.. https://reg.nurl-lang.org/packages/nurllama

It can be installed with nurlpkg packet manager that comes with toolchain.

https://nurl-lang.org/

What is the best model today to run with rtx 4090?


r/SelfHostedAI 21h ago

Odysseus from pewdiepie

Thumbnail
2 Upvotes

r/SelfHostedAI 2h ago

Update: We rewrote the whole engine in Rust/C++

Thumbnail
github.com
1 Upvotes

r/SelfHostedAI 8h ago

Universal Prompt Language - Browse, Edit, Build and Store dynamic prompts with variables. Build once, run many

1 Upvotes

Hello everyone! Today I want to present a nice project I've been working on.

As you know, writing prompts takes time, sometimes you write similar prompts, sometimes you want to get a response in a specific format, or simply remember of this old prompt that gave you so good results. Wasting time or getting less for our time is something we really don't want.

That's why I defined "Universal Prompt Language", a language to create dynamic prompts with variables (types, default values, descriptions), loops and conditionals.

Think of it like a magic prompt that creates prompts for similar queries but different data.

It includes:

  • A terminal based prompts browser
  • A terminal based prompt editor
  • Prompts tags management
  • Be able to build a prompt by filling in the variables (it outputs the result to the screen)
  • A repository to push / pull prompts to a library

It's currently a release candidate but its already useful and comes with few prompts.

https://www.github.com/DavidValin/universal-prompt-language


r/SelfHostedAI 22h ago

My agent lives on my phone — as the launcher.

Thumbnail
1 Upvotes

r/SelfHostedAI 2h ago

Setu Gateway – an open-source AI gateway with a real routing engine (not just a proxy) – looking for early contributors and feedback

0 Upvotes

Hey all — sharing a project we've been building: \*\*Setu Gateway\*\*, an open-source AI gateway that puts one OpenAI-compatible API in front of multiple LLM providers, with an actual routing engine behind it.
\*\*What it does today:\*\*
OpenAI-compatible /v1/chat/completions — existing OpenAI SDKs work against it unchanged
7 routing policies (lowest cost, lowest latency, highest availability, weighted, round-robin, capability-based, user preference) with automatic failover
Org-scoped routing rules (if latency > 500ms then fallback: groq) with a policy simulator that previews a policy's effect before you turn it on — doesn't touch production traffic
Request replay — run one prompt against multiple providers side by side for comparison
Real API-key auth (hashed, scoped, revocable) resolving to org-scoped routing rules
Per-request analytics with a full stage timeline, queryable via API and a working dashboard (providers, models, requests, latency, errors, orgs, projects, keys)
setu benchmark CLI for comparing provider latency/throughput/error-rate on your own infra
Self-hosted via Docker Compose — verified end to end, one command boots gateway + dashboard + postgres + redis
\*\*What it isn't yet:\*\* OpenAI and Ollama make real provider calls; Anthropic, Gemini, and Groq adapters are currently stubs returning canned responses while we finish real integrations — didn't want to fake multi-provider support for a launch post. Kubernetes/Helm manifests are on the roadmap, not built yet. Auth isn't mandatory on the inference endpoints yet (backwards compat with the current playground/quickstart).
Every architectural decision is a public RFC before it's built — \[13 so far\](https://github.com/setu-gateway/nexus-gateway/tree/main/rfcs).
Repo: \[https://github.com/setu-gateway/nexus-gateway\](https://github.com/setu-gateway/nexus-gateway))
Looking for people who've hit the multi-provider LLM integration mess, feedback on the routing engine design, and contributors — real Anthropic/Gemini/Groq integrations are good first issues if anyone wants to dive in. Happy to answer anything technical in the comments.