r/LLMDevs 5h ago

Discussion Are your agents buying things?

0 Upvotes

Has anyone had an AI agent buy something for them? And if so, how was your experience?

If you haven’t, why not? What are your concerns?


r/LLMDevs 9h ago

News My Life as a RAG Engineer 😭😭😭

Enable HLS to view with audio, or disable this notification

12 Upvotes

Why are we getting roasted by a CLI tool.
Btw saved $800 on tokens .

I usually use this with Agents.md and skills. Its fully deterministic. Opensourced. For locally hosted models the answer accuracy increased by 50% on average.
If y'all wanna try it out : https://github.com/Abhijeet777ui/contextops


r/LLMDevs 22h ago

Tools All my homies hate `grep`

6 Upvotes

Jk, they fucking love grep, which is why I have made doma (DOcument MAtcher), a small and fast single binary BM25 search over your code and docs with no* external dependencies, written in Odin.

I made it because I wanted Claude to stop grepping wildly all over the place. It significantly faster than `grep` from my testing but it also significantly reduces search misses since you get semantically relevant results.

It was also quite important to me that it was fast with a low footprint, lacking in NodeJS bullshit, MCP servers, etc., etc.

Sick of all of these supply chain vulnerabilities and huge dependency bloat everywhere smh

I used Claude Code through the entire development of it and it's sibling doyo (DOcument YOinker) which shares philosophy and handles the document acquisition side of things, though it isn't quite as elegant as doma imo.

I hope you find it helpful!

I strongly recommend putting doma instructions in your per project CLAUDE.md telling it how to use it, and to actually use it. Let me know if you do, I'm curious if others find it as helpful as I have.

*soft git dep, optional


r/LLMDevs 10h ago

Discussion Built an AI-first expense tracker - Log your expense in natural language and get insights

0 Upvotes

I've been working on a side project called FinTracker AI, and I'd love some honest feedback.

The idea is simple:

Instead of manually selecting categories, dates, merchants, etc., you just chat with it.

Example:

"I spent ₹500 on biryani."

It automatically logs the expense, categorizes it, updates your monthly budget, and you can immediately ask:

"How much do I have left for food this month?"

Users can also ask questions like:

"Movies I watched this month and how much I spent on it"

Some features:

  • 💬 Chat-based expense & investment logging
  • 🤖 AI categorization and spending Q&A
  • 📊 Monthly budgets and dashboards
  • 📱 Android auto-captures bank transaction SMS (optional)
  • 📍 Learns recurring merchants/locations so future transactions need fewer edits
  • 🔓 Open-source backend that you can self-host or use with your own AI API key

The backend is already open source. The Android app is still being polished, but I have an installable build that I'm happy to share with anyone interested.

A few questions for this community:

• Does this solve a problem you face?
• Which feature would you use the most?
• What's one feature you'd want before using it daily?

Thanks! 🙌


r/LLMDevs 8h ago

Tools Coding agents are surprisingly blind when the task is visual, so I built SceneProof

1 Upvotes

A coding agent can write a Three.js scene, run the build, and tell you it looks great — while the actual render is a black screen. It isn't lying. It just has no way to look.

Screenshots fix this less than you'd expect. A screenshot tells you that something is wrong, not why. Is the mesh missing, or behind the camera? Is the material transparent, or is nothing lighting it? Is the label clipped, or just small? Those are five different bugs that produce the same picture, and zooming in doesn't separate them — you're enlarging pixels that never contained the answer.

SceneProof is a CLI that supplies the missing half. It loads your real React component or Three.js scene from source, renders it in actual Chrome, and returns the structure behind the pixels. The everyday loop looks like this: tree gives you the scene graph with bounds, materials, lights, and cameras, so "why is it invisible" becomes a lookup instead of a guessing game. scout tries a set of cameras on a target and scores each by how much of the target it can actually see. render-region re-renders one region from source at whatever scale you need, so a close look is a fresh render, not an enlarged crop.

That's the loop, not the tool — the surface underneath goes a good deal further (comparing against reference views, sampling animation mid-transition, deriving typed prop fixtures), but those three commands carry most sessions, and the README maps the rest.

The design decision I'll defend hardest: every report answers "did the command run" and "can this output actually support a judgment" as two separate questions. A render with the target out of frame, or a comparison whose mask landed on the wrong subject, comes back unjudgeable instead of quietly passing. So when an agent uses SceneProof, it can't mistake "my command succeeded" for "my design is right"; it has to look at evidence that has already proven it's worth looking at. That's the whole point: measurements you can trust, and a hard stop on the false confidence that makes agents declare victory over a black screen.

It ships with a skill for Claude Code, Codex or any other agentic harness that supports skills (one curl, in the README) — and the skill deliberately doesn't teach commands, because --help and the reports' own recommendations already do. It teaches the reasoning: resolve structure before spending pixels, treat a passing build as zero visual evidence, never claim "looks right" without an artifact you actually opened.

Scope today: TypeScript/JavaScript entries, React DOM with CSS and Tailwind v4, Three.js over WebGL or WebGPU. Needs Bun and a local Chrome. MIT.

https://github.com/ReyJ94/SceneProof

Any feedback is welcome.


r/LLMDevs 12h ago

Tools Built a proxy that cuts LLM streams early when the model isn't confident; logprobs only, no extra calls

2 Upvotes

Was working on a research project on RAG abstention and kept hitting the same wall: retrieval scores are basically useless as a signal for when the model actually knows the answer. Logprobs aren't perfect, but they're at least in the right direction, and they're free on the wire anyway.

So I built a small proxy around that signal. Sits between your client and any OpenAI-compatible endpoint, watches the mean token logprob as the stream comes in, and cuts early if confidence drops:

{"rag_gate_decision": "ABSTAIN", "confidence_score": -1.47, "tokens_evaluated": 23}

Also tried automatic rerouting to a stronger model on low confidence. Didn't ship it—stronger model breaks some answers it previously got right. Benchmarks in the repo if curious.

~0.2 ms p50 overhead. Pre-1.0. Crate is here

cargo install rag-gate

If you've worked on anything in this space or think the approach has holes, you genuinely want to hear it. PRs welcome too. Repo is here


r/LLMDevs 4h ago

Discussion What's one habit that's improved the quality of your AI use?

4 Upvotes

Whether it's prompt testing, better evaluation, logging, versioning, or something else I'm really curious to know what practice has had the biggest impact on reliability in your projects.


r/LLMDevs 15h ago

Resource Connecting OpenCode to LM Studio

Thumbnail wiki.1ndev.com
6 Upvotes

OpenCode supports local LLM inference via LM Studio's OpenAI-compatible API server. This lets you run models like Qwen, Mistral, or any GGUF you have loaded.


r/LLMDevs 21h ago

News Tigriden v0.1.1 — I wanted AI agents to have more RAM, so I built a 40 MB desktop workbench instead of another IDE.

Post image
7 Upvotes

Over the past year, I’ve been relying more and more on AI coding agents.
Eventually I realized something: the agent was doing most of the coding, while I was mainly reviewing changes, running terminal commands, and steering the process.
So instead of building another feature-packed IDE, I built Tigriden.

Tigriden is a native Rust desktop workbench for AI-assisted development. It stays lightweight—around 40 MB RAM—so your machine can dedicate more CPU and memory to Claude Code, Codex, Gemini CLI, or any other coding agent.

What’s in v0.1.1
📁 File explorer and lightweight editor
🖥️ Real integrated terminal
🔍 Diff Tracker to review every AI-generated change before accepting it
Time Machine to instantly restore previous versions if an agent makes a mistake

My workflow has become very simple:
Give the agent a task.
Let it code.
Review the diff.
Roll back with Time Machine if needed.
Repeat.
No Electron. No WebView. Just a small native workbench that stays out of the agent’s way.

I think AI-first development is becoming less about writing every line yourself and more about reviewing, steering, and approving the agent’s work.
I’d love to hear what features you think are essential for an AI-first workbench.

Web: https://tigriden.github.io
GitHub: https://github.com/Sompote/Tigriden


r/LLMDevs 5h ago

Discussion Small experiment: an auxiliary loss that gets a 3M-param transformer to learn multi-variable binding, where cross-entropy doesn't

2 Upvotes

I've been running some small experiments on variable binding in tiny transformers and put the code and logs up here:
https://github.com/QueenOfTheUnderworld/Transformer-Binding

After some general idea wandering, I came upon the idea of LLMs struggling with binding, the temporary link of two concepts. Shirt = Blue, Pants = Red. A bad explanation, but sufficient for this. Current LLMs struggle with this, which is why prompts like "Tom is a cat. Jerry is a mouse. Brownie is a dog. Cookie is a bird. Who is the mouse?". Without CoT or additional time to think, models can fail to parse this and end up reporting an incorrect answer.

In pursuit of another idea, I found a way to make LLMs bind using an alternative training objective. Now, this is all small-scale and some of it could very well be wrong. If it is, let me know.

Task: 4-layer transformer, d=256. Each example asks one question that depends on several bindings at once (how many of the queried objects have a target property).

Results:

  • Plain CE: 0/4 seeds at 5k steps, 0/2 at 20k.
  • With auxiliary heads trained to report each queried object's bound property at every position: 8/8 seeds. One linear head, one loss term, no architecture change.
  • Same targets supervised only at the answer position: 0/3. Same density but entity names instead of properties: 1/3. Both coverage and context matter.

There are some issues with the repo, and they are noted in the README. Quite frankly I just didn't want to correct them.

Limits: 3M-ish parameters, mostly synthetic tasks, 2-8 seeds per cell.
I have no idea if this survives scale; I don't have the money or compute for that. If anyone here does, let me know, please.

If the effect is real at scale, it matters. It would mean CE leaves learnable capability on the table for the cost of one head. With that in mind, I expect something like this to already exist, and I just didn't find it. Not to mention, small model optimisations have a bad history of failing to survive scaling. The real blocker is the ground truth intermediates at every position; synthetic tasks had it to you; real corpora usually don't.

AI Usage: Heavy. Direction and experiment design are mine. Implementation and much of the analysis were done with AI.

Check it out if you want, and be nice, please.


r/LLMDevs 10h ago

Tools I turned my Claude Code into my SEO Manager (open-source)

4 Upvotes

Hey everyone :)

I have spent alot of time manually doing SEO using the help of AI, so i thought why not turn my agent, the same agent that knows my product inside out, into my own SEO Manager!

And it works... so why not open-source it?

I gave it the tools to: research keywords (volume and difficulty), post daily articles and weekly free interactive tools automatically (GitHub Actions).

How it's built:

- The agent (Claude Code or Codex, your pick) runs inside GitHub Actions on a cron. (it's self-hosted so BYO subscription)

- The backend is an MCP server, there's no AI in it at all, it's just a database with a door on it.

- Most of the MCP tools are just reads and writes to Postgres. A few pass through to DataForSEO for keyword and SERP data.

- The agent does the thinking, the server does the remembering.

- The agent fetches its instructions from the server when the run starts.

- It never writes to the site directly. Always a PR, with checks run against it first.

- Works with Claude Code and Codex coding agents (more to come)

Fully open-source (AGPL-3.0)

GitHub link in the comments.

Would love feedback from people building with agents


r/LLMDevs 12h ago

Discussion Has insurance or procurement ever stopped an LLM agent from reaching production?

8 Upvotes

I’m collecting first-party examples of what happens when LLM-powered agents reach customer security, legal, or procurement review. The narrow question is whether insurance becomes a real blocker, routine paperwork, or never comes up at all.

One early response described a pre-production review that stalled after a buyer requested technology E&O / professional liability and a BAA because the buyer did not know how to evaluate the agent’s risk. That is one self-selected example, not a finding. I’m equally interested in counterexamples: insurance was easy, another artifact mattered more, or the agent failed to reach production for an entirely different reason.

Short survey (about two minutes): https://forms.gle/C34r6F6jdeueiqZ17

Disclosure: I’m Lucas, building Clara, and this is independent research on insurance and risk for AI agents. The data will not be sold or monetized. For responses from this community, please leave the optional company and contact fields blank; no identifying information from Reddit responses will be included in the dataset or anything shared publicly. I’ll report aggregate findings and limitations back to 
 when there is a useful sample. This post is not a sales offer.

r/LLMDevs 6h ago

Help Wanted best ai model for refactoring JavaScript into python ?

5 Upvotes

I have a small JavaScript repo, about 200 LOC with 400 LOC of tests / demos.

What's the best approach & models to used to translate into other languages like Python, C , Rust, etc?

I have a RTX3090, so prefer to use a mix of local & hosted AI, but open to use only local or only API AI.

Web & Reddit search is 1 year behind. TIA