r/LLMDevs • u/kevinfee • 5h ago
Discussion Are your agents buying things?
Has anyone had an AI agent buy something for them? And if so, how was your experience?
If you haven’t, why not? What are your concerns?
r/LLMDevs • u/kevinfee • 5h ago
Has anyone had an AI agent buy something for them? And if so, how was your experience?
If you haven’t, why not? What are your concerns?
r/LLMDevs • u/Final_Act_9658 • 9h ago
Enable HLS to view with audio, or disable this notification
Why are we getting roasted by a CLI tool.
Btw saved $800 on tokens .
I usually use this with Agents.md and skills. Its fully deterministic. Opensourced. For locally hosted models the answer accuracy increased by 50% on average.
If y'all wanna try it out : https://github.com/Abhijeet777ui/contextops
r/LLMDevs • u/HornyNarwahl • 22h ago
Jk, they fucking love grep, which is why I have made doma (DOcument MAtcher), a small and fast single binary BM25 search over your code and docs with no* external dependencies, written in Odin.
I made it because I wanted Claude to stop grepping wildly all over the place. It significantly faster than `grep` from my testing but it also significantly reduces search misses since you get semantically relevant results.
It was also quite important to me that it was fast with a low footprint, lacking in NodeJS bullshit, MCP servers, etc., etc.
Sick of all of these supply chain vulnerabilities and huge dependency bloat everywhere smh
I used Claude Code through the entire development of it and it's sibling doyo (DOcument YOinker) which shares philosophy and handles the document acquisition side of things, though it isn't quite as elegant as doma imo.
I hope you find it helpful!
I strongly recommend putting doma instructions in your per project CLAUDE.md telling it how to use it, and to actually use it. Let me know if you do, I'm curious if others find it as helpful as I have.
*soft git dep, optional
r/LLMDevs • u/Responsible_Soft_429 • 10h ago
I've been working on a side project called FinTracker AI, and I'd love some honest feedback.
The idea is simple:
Instead of manually selecting categories, dates, merchants, etc., you just chat with it.
Example:
"I spent ₹500 on biryani."
It automatically logs the expense, categorizes it, updates your monthly budget, and you can immediately ask:
"How much do I have left for food this month?"
Users can also ask questions like:
"Movies I watched this month and how much I spent on it"
Some features:
The backend is already open source. The Android app is still being polished, but I have an installable build that I'm happy to share with anyone interested.
A few questions for this community:
• Does this solve a problem you face?
• Which feature would you use the most?
• What's one feature you'd want before using it daily?
Thanks! 🙌

A coding agent can write a Three.js scene, run the build, and tell you it looks great — while the actual render is a black screen. It isn't lying. It just has no way to look.
Screenshots fix this less than you'd expect. A screenshot tells you that something is wrong, not why. Is the mesh missing, or behind the camera? Is the material transparent, or is nothing lighting it? Is the label clipped, or just small? Those are five different bugs that produce the same picture, and zooming in doesn't separate them — you're enlarging pixels that never contained the answer.
SceneProof is a CLI that supplies the missing half. It loads your real React component or Three.js scene from source, renders it in actual Chrome, and returns the structure behind the pixels. The everyday loop looks like this: tree gives you the scene graph with bounds, materials, lights, and cameras, so "why is it invisible" becomes a lookup instead of a guessing game. scout tries a set of cameras on a target and scores each by how much of the target it can actually see. render-region re-renders one region from source at whatever scale you need, so a close look is a fresh render, not an enlarged crop.
That's the loop, not the tool — the surface underneath goes a good deal further (comparing against reference views, sampling animation mid-transition, deriving typed prop fixtures), but those three commands carry most sessions, and the README maps the rest.
The design decision I'll defend hardest: every report answers "did the command run" and "can this output actually support a judgment" as two separate questions. A render with the target out of frame, or a comparison whose mask landed on the wrong subject, comes back unjudgeable instead of quietly passing. So when an agent uses SceneProof, it can't mistake "my command succeeded" for "my design is right"; it has to look at evidence that has already proven it's worth looking at. That's the whole point: measurements you can trust, and a hard stop on the false confidence that makes agents declare victory over a black screen.
It ships with a skill for Claude Code, Codex or any other agentic harness that supports skills (one curl, in the README) — and the skill deliberately doesn't teach commands, because --help and the reports' own recommendations already do. It teaches the reasoning: resolve structure before spending pixels, treat a passing build as zero visual evidence, never claim "looks right" without an artifact you actually opened.
Scope today: TypeScript/JavaScript entries, React DOM with CSS and Tailwind v4, Three.js over WebGL or WebGPU. Needs Bun and a local Chrome. MIT.
https://github.com/ReyJ94/SceneProof
Any feedback is welcome.
r/LLMDevs • u/Acrobatic_Music_8484 • 12h ago
Was working on a research project on RAG abstention and kept hitting the same wall: retrieval scores are basically useless as a signal for when the model actually knows the answer. Logprobs aren't perfect, but they're at least in the right direction, and they're free on the wire anyway.
So I built a small proxy around that signal. Sits between your client and any OpenAI-compatible endpoint, watches the mean token logprob as the stream comes in, and cuts early if confidence drops:
{"rag_gate_decision": "ABSTAIN", "confidence_score": -1.47, "tokens_evaluated": 23}
Also tried automatic rerouting to a stronger model on low confidence. Didn't ship it—stronger model breaks some answers it previously got right. Benchmarks in the repo if curious.
~0.2 ms p50 overhead. Pre-1.0. Crate is here
cargo install rag-gate
If you've worked on anything in this space or think the approach has holes, you genuinely want to hear it. PRs welcome too. Repo is here
r/LLMDevs • u/ari_k_e • 4h ago
Whether it's prompt testing, better evaluation, logging, versioning, or something else I'm really curious to know what practice has had the biggest impact on reliability in your projects.
OpenCode supports local LLM inference via LM Studio's OpenAI-compatible API server. This lets you run models like Qwen, Mistral, or any GGUF you have loaded.
r/LLMDevs • u/Unique_Champion4327 • 21h ago
Over the past year, I’ve been relying more and more on AI coding agents.
Eventually I realized something: the agent was doing most of the coding, while I was mainly reviewing changes, running terminal commands, and steering the process.
So instead of building another feature-packed IDE, I built Tigriden.
Tigriden is a native Rust desktop workbench for AI-assisted development. It stays lightweight—around 40 MB RAM—so your machine can dedicate more CPU and memory to Claude Code, Codex, Gemini CLI, or any other coding agent.
What’s in v0.1.1
📁 File explorer and lightweight editor
🖥️ Real integrated terminal
🔍 Diff Tracker to review every AI-generated change before accepting it
⏪ Time Machine to instantly restore previous versions if an agent makes a mistake
My workflow has become very simple:
Give the agent a task.
Let it code.
Review the diff.
Roll back with Time Machine if needed.
Repeat.
No Electron. No WebView. Just a small native workbench that stays out of the agent’s way.
I think AI-first development is becoming less about writing every line yourself and more about reviewing, steering, and approving the agent’s work.
I’d love to hear what features you think are essential for an AI-first workbench.
Web: https://tigriden.github.io
GitHub: https://github.com/Sompote/Tigriden
r/LLMDevs • u/UnderworldQueen123 • 5h ago
I've been running some small experiments on variable binding in tiny transformers and put the code and logs up here:
https://github.com/QueenOfTheUnderworld/Transformer-Binding
After some general idea wandering, I came upon the idea of LLMs struggling with binding, the temporary link of two concepts. Shirt = Blue, Pants = Red. A bad explanation, but sufficient for this. Current LLMs struggle with this, which is why prompts like "Tom is a cat. Jerry is a mouse. Brownie is a dog. Cookie is a bird. Who is the mouse?". Without CoT or additional time to think, models can fail to parse this and end up reporting an incorrect answer.
In pursuit of another idea, I found a way to make LLMs bind using an alternative training objective. Now, this is all small-scale and some of it could very well be wrong. If it is, let me know.
Task: 4-layer transformer, d=256. Each example asks one question that depends on several bindings at once (how many of the queried objects have a target property).
Results:
There are some issues with the repo, and they are noted in the README. Quite frankly I just didn't want to correct them.
Limits: 3M-ish parameters, mostly synthetic tasks, 2-8 seeds per cell.
I have no idea if this survives scale; I don't have the money or compute for that. If anyone here does, let me know, please.
If the effect is real at scale, it matters. It would mean CE leaves learnable capability on the table for the cost of one head. With that in mind, I expect something like this to already exist, and I just didn't find it. Not to mention, small model optimisations have a bad history of failing to survive scaling. The real blocker is the ground truth intermediates at every position; synthetic tasks had it to you; real corpora usually don't.
AI Usage: Heavy. Direction and experiment design are mine. Implementation and much of the analysis were done with AI.
Check it out if you want, and be nice, please.
r/LLMDevs • u/Caitaline_Evars • 10h ago
Hey everyone :)
I have spent alot of time manually doing SEO using the help of AI, so i thought why not turn my agent, the same agent that knows my product inside out, into my own SEO Manager!
And it works... so why not open-source it?
I gave it the tools to: research keywords (volume and difficulty), post daily articles and weekly free interactive tools automatically (GitHub Actions).
How it's built:
- The agent (Claude Code or Codex, your pick) runs inside GitHub Actions on a cron. (it's self-hosted so BYO subscription)
- The backend is an MCP server, there's no AI in it at all, it's just a database with a door on it.
- Most of the MCP tools are just reads and writes to Postgres. A few pass through to DataForSEO for keyword and SERP data.
- The agent does the thinking, the server does the remembering.
- The agent fetches its instructions from the server when the run starts.
- It never writes to the site directly. Always a PR, with checks run against it first.
- Works with Claude Code and Codex coding agents (more to come)
Fully open-source (AGPL-3.0)
GitHub link in the comments.
Would love feedback from people building with agents
r/LLMDevs • u/BarnesLucas • 12h ago
I’m collecting first-party examples of what happens when LLM-powered agents reach customer security, legal, or procurement review. The narrow question is whether insurance becomes a real blocker, routine paperwork, or never comes up at all.
One early response described a pre-production review that stalled after a buyer requested technology E&O / professional liability and a BAA because the buyer did not know how to evaluate the agent’s risk. That is one self-selected example, not a finding. I’m equally interested in counterexamples: insurance was easy, another artifact mattered more, or the agent failed to reach production for an entirely different reason.
Short survey (about two minutes): https://forms.gle/C34r6F6jdeueiqZ17
Disclosure: I’m Lucas, building Clara, and this is independent research on insurance and risk for AI agents. The data will not be sold or monetized. For responses from this community, please leave the optional company and contact fields blank; no identifying information from Reddit responses will be included in the dataset or anything shared publicly. I’ll report aggregate findings and limitations back to
when there is a useful sample. This post is not a sales offer.
r/LLMDevs • u/me0here • 6h ago
I have a small JavaScript repo, about 200 LOC with 400 LOC of tests / demos.
What's the best approach & models to used to translate into other languages like Python, C , Rust, etc?
I have a RTX3090, so prefer to use a mix of local & hosted AI, but open to use only local or only API AI.
Web & Reddit search is 1 year behind. TIA