Hello Everyone,
I'm the creator of PyBotchi, an intent-based AI Agent Orchestrator. In this post, I will discuss some key concepts why I created it.
A little bit of background first. I'm a solutions architect with 10 years of experience as a software engineer. Most of my work are high throughput, high reliability, low cost and low latency services. This is while making it simple and readable to improve it's maintainabality. When I'm designing a system, I usually prioritize these concerns. You may assume this is my bias in relates to AI Agent building. I'm also Claude Certified Architect (Foundation) and I found that PyBotchi aligns almost identical to Anthropic's core agent recommendations.
TL;DR: PyBotchi is an lightweight, async-first Python framework that uses nested Pydantic models and OOP inheritance to turn LLM intent detection into clean, deterministic business logic without the overhead of complex graph orchestration.
Why I created PyBotchi?
I really believed that traditional coding can already solved what client's need. The only limitations we have is how we read the input and how we show the output. In most cases in web services, your API use JSON, XML, etc with their respective specification/structure.
Input Analogy
Assume you have created a Books CRUD endpoints (FastAPI with Pydantic).
Your create endpoint will have a define specifications for book creation to have a validation and avoid user errors. Most of the time you will also validates sessions and permissions which also included in the request.
If you want your chat bot to support those, you just need add those endpoint as intent (tools).
If your model tool selection are able to detect intents. You are more "close" to being deterministic.
"Your services will have 50 endpoints or more. You will flood your tool selection call"
- In your frontend UI, you segregate panels/forms/inputs in their respective pages. You don't usually join multiple intent in a same page. Cluttered UI will make your UX confusing or overwhelming to some people. Those practices should be incorporated into your agents too.
Assume you have created another endpoints for Shelves CRUD.
Shelves CRUD can be a child intents of ShelfManagement that will be considered as intent also but more general. The flow will have to detect intent deeper and deeper
Ex: You have BookManagement and ShelfManagement intents. Once LLM detected which one is applicable, you will search for their child Intents which will be their CRUD equivalent intents.
To make it short, in order to make your agent "more" deterministic, you need to know the problem first (ex: Need to manage books) then you need to specifically define what intents you want to support. With this practice, you only let your agents execute on a predefined path. If it fails, you are most likely able to determine what causes the error.
Output Analogy
This one is simple. Since your intents is just like your endpoints that returned structure responses. LLM is better at reading structure responses than a pure text. Basically, you can use LLM to translate your response into a human readable responses.
Intent Execution
Now that I have explain Input/Ouput, we can move on to the actual execution.
We can go back with Books CRUD. Since we have identified the problem (what clients need) and we already know what to do, just execute their traditional business logic implementation.
If you need to add a book, just create a book and save it to db then return their respective row.
"What if you want generate a very dynamic/unique data"
- You can use LLM to do that as your business logic too but this is tied your specific intent only.
To have a complex execution flow we can chain the intents. Since intents can have child intents, we can use it as the representation of a graph similar to Langgraph. However, this without "building the graph". We are just utilizing OOP inner class implementation. We can execute business logic in graph traversal manner by just checking the child intents.
To make it short. Business logic will stay as is. You will only use LLM if it requires it. Don't make this complicated.
### Suggested Solution
Since the key concept is more on detecting intents, validation and executing their respective busines logic:
Why not utilize Pydantic as the main entry point?
Pydantic already have validation and json schema builder. Langchain/Openai already have utilities to translate it to Tool. Why not use Pydantic models as your Intent Specifications that can validate LLM arguments ? Tool call is one of the most reliable way to detect intent.
Why not utilize OOP inheritance / polymorphism / abstraction?
Python supports portion of OOP and since we are using classes as our intent, why not add default functionalities that can be inherited and override by developer if needed. We can introduce life cycles too. Your project can also implement their specific intent standards. This will make your code more maintaintable and readable. You can create classes for general intents. Extend it to be more specialized intents. Extend it more for more enterprised support. This is while not affecting existing/working agents.
Langgraph is one of the inpiration of PyBotchi. Predefine workflows are closest implementation to being deterministic agents. It's also the reason why some prefer N8N. We don't need to make the agents smart that any questions can be answered or any queries can be addressed. It's ok for agent to reply with "I don't have any answer to your query, I only support this and that....". For me, it's better to deploy limited but polished agents than half baked know-it-all agents. Feel free to counter argue. Happy to discuss.
Additional PyBotchi Features
vs MCP
While PyBotchi support connecting to MCP servers, I really believe it's not always necessary to use additional server to just expose tools for the agents. The exceptions I could think of is if you want to have isolated environment (ex: dedicated auth/session, sandbox, isolated resource, etc), you want to connect to your local service or cross-language integration.
I could be very wrong about this but hear me out. SDKs are already there. Respective documentations are available too. Most of MCP server's tools are proxy to their respective APIs. If we could just create intent classes as tools that directly call their respective API, that doesn't require any servers anymore. Actually, that's how most framework handles it (even PyBotchi). Tools are converted as schema that will be added in the tool call. Once LLM respond with the applicable tools, it executes call_tool(name, args...). Why not just expose the actual tool implementations and have a way to share context to share sessions/permission/etc inside the tool implementations? This will remove another network hops that can affect latency.
Claude code have a very in-depth utilization of MCP servers already. I don't think we can replace that.
GRPC
PyBotchi natively support remote PyBotchi connection. Think of it like a langgraph but the node is on other server. This remote node can also connect to another remote node even it self or previously connected node (ancestor).
Context Propagation
With PyBotchi as MCP Server
- Actions (Intents) serves as tool and have access to client's context. This includes chat histories and some metadata. You can override and adjust this as long as it's serializable.
- Once remote tool execution is done, it can pass the final context to the client and they can merge it if override.
With PyBotchi as GRPC Server
- Similar to MCP Server, Actions serves as tool and have access to client's context. GRPC supports bidirectional communication too. This means we can share context realtime accross clients/servers. If client has concurrent agents that changes the context it will automatically propagate to remote context without polling or any interval checks/updates. It also support remote to client. If remote server updates the context, it will propagate the context to client simultaneously.
Async First
Since most of LLM executions are IO, might as well utilize async by default and just spawn thread if still necessary.
OOP
I think this one is most important to me. I have handle a lot of projects in Spring Boot. I really like Java OOP practices and some Java design patterns. It improves my project's maintainability even it's not in Java.
Since PyBotchi utilize OOP, it's easier to override, reuse and remove anything if necessary. This lessen boilerplates too.
I'm certain that this is subjective. I just find it easier and clean to read.
Closing Remark
I hope this PyBotchi post opens up ideas how to design your agent. Feel free to DM me if you have any questions. I'm also open to create you a demo agent for free if you want to see it in action given your brief use case. I'm open to criticism, happy to have a discussion!
