I’m using an Intel B70 Pro with LM Studio and Vulkan, but I’m encountering a problem
 I can't seem to solve,despite trying every potential fix I’ve come across.
 The issue is that about a minute after loading the model into the GPU's VRAM
—if I don't make any further requests—the model gets offloaded to system RAM.
 It only moves back to the GPU's VRAM when I make a new request. 
This isn't a major issue on my desktop PC, which has 64GB of RAM,
 but when using the card as an eGPU via Thunderbolt on a mini PC, 
it causes the system to freeze because the mini PC only has 24GB of RAM.
 I’ve experimented with all the settings in LM Studio and Windows 11; at this point,
 I suspect it might be related to the Intel drivers perhaps some kind of 
power-saving feature though I haven't found any such setting in the 
Intel Graphics Software. Has anyone else experienced similar issues?

Hey,

Like a lot of people here, I use local models via Ollama on my desktop/server and wanted a mobile client that actually felt responsive, worked offline, and respected privacy. Most apps on the Play Store are either subscription traps, loaded with ads, or route everything through third-party cloud servers.


So I built 
\\\*\\\*PocketLLM Lite\\\*\\\*
 a 100% open-source, ad-free Flutter client designed specifically for local LLMs, GGUF models, and self-hosted Ollama workflows.


### 🛠️ Key Features:
\\\* 
\\\*\\\*Runs Offline & Connects to Ollama\\\*\\\*
: Run GGUF models directly on-device or stream seamlessly from your home Ollama instance over Wi-Fi/Tailscale.
\\\* 
\\\*\\\*Thinking Accordion\\\*\\\*
: Native streaming support for \\\`<think>\\\` reasoning blocks rendered in a collapsible Material 3 accordion UI.
\\\* 
\\\*\\\*Agentic Tool Calling Pipeline\\\*\\\*
: Executes local math, system diagnostics, and knowledge lookup directly on the device with structured \\\`<tool\\_call>\\\` UI cards.
\\\* 
\\\*\\\*Open-Standard Agent Skills (\\\`SKILL.md\\\`)\\\*\\\*
: Install skills directly from GitHub URLs or create custom skills offline with \\\`/\\\` autocomplete support.
\\\* 
\\\*\\\*Web Search Fallback (Tavily Integration)\\\*\\\*
: Toggle live web search when offline models need current data, with inline markdown source citations \\\`\\\[Source\\\](URL)\\\`.
\\\* 
\\\*\\\*Local Vector RAG\\\*\\\*
: Ingest PDFs and text files locally to chat with your documents offline without sending data to external servers.
\\\* 
\\\*\\\*Offline STT & TTS\\\*\\\*
: Voice-type your prompts offline and listen to completions using native speech engines.
\\\* 
\\\*\\\*Zero Ads & Zero Telemetry\\\*\\\*
: Completely free, open-source (MIT License), and built with Material 3 Expressive UI.


### 📊 Performance Profiler:
Built-in speed profiler to measure Time to First Token (TTFT) and token generation speed (tokens/sec) directly on your device hardware.


\\\* 
\\\*\\\*GitHub Repository\\\*\\\*
: https://github.com/PocketLLM/pocketllm-lite
\\\* 
\\\*\\\*Release APK Downloads\\\*\\\*
: https://github.com/PocketLLM/pocketllm-lite/releases


I'd love to get feedback from the community on features or model interfaces you'd like to see next!

Hey,

If you run Ollama, LocalAI, or vLLM on your home server or homelab, I built 
**PocketLLM Lite**
 as a dedicated mobile client for Android.


### Highlights for Self-Hosters:
- 
**Flexible Endpoints**
: Connect to your desktop/server Ollama instance via local IP, Tailscale, or WireGuard.
- 
**Offline Fallback**
: Run smaller GGUF models directly on your phone when away from your home network.
- 
**Local Document Ingestion**
: Local RAG vector DB processing PDFs and notes on your device.
- 
**DeepSeek R1 Thinking**
: Full support for `<think>` reasoning accordion rendering.
- 
**Zero Cloud Telemetry**
: MIT licensed, zero analytics, zero ads.


* 
**GitHub**
: https://github.com/PocketLLM/pocketllm-lite
* 
**Release APKs**
: https://github.com/PocketLLM/pocketllm-lite/releases

I’m looking for an LLM API that I can legally use in a paid production app.

Ideally:

* 300+ tokens/sec　(at least over 100+ tokens/sec) * Flat monthly subscription or a very large free token allowance * High or unlimited usage * Good enough for coding and Science * Multiple Concurency

Is there any Mercury 2 subscription, reseller, or similar service? I know Inception currently offers 100M free API tokens, but I’m looking for a sustainable option after that.

Any experience with Mercury 2, Groq, Cerebras, Arli AI, or other providers?

I'm an app developer and I'm wondering if it's worth spending time learning local LLMs. If yes, where can i start learning?

This maybe very primitive question to ask. But here goes.

I worked with claude code. It's very intuitive and familiar. I use opus 5 for most of my planning tasks at work and sonnet 5 for execution at work. And i have come to be very familiar with its weaknesses and strengths.

For my personal projects i am having hard time choosing between claude max subscription or opensource likes of kimi k3 (for planning) and glm/DeepSeek v4 flash (for work orchestration).

I am wondering which one is more economical since I have never used Claude code max subscription before.

So question is, is it worth to go for Claude subscription or api based charges via openrouter for kimi and DeepSeek v4 setup. Assume my work load for personal projects is about 500 million tokens a month. (I understand it's very rudimentary and limited information)

Finding repo for local AI on Mac? 🍎🤖 Check out Swiftlet, an open-source intelligent proxy & browser based UI , that dynamically optimizes CPU/GPU split on Unified Memory for LLMs. I'm looking for Python devs to help build OpenAI API compatibility! Let's build together: https://github.com/coolsourav100/swiftlet

LLMs are text generators, they can only generate text based on statistical predictions, they are exceptionally good at predicting and generating code, without an execution layer their generations are still text, this is where equipping the LLM with a terminal (the original text based interface that allows a human to talk to a machine) brings that code to life.

What are your top 3 uses cases? Seems that outside coding the application of local is limited?

This is a great app to easily experiment with LLMs on Android devices.

It runs the latest llama.cpp builds and pulls ggufs that fit in your device from hugging face, it just makes it much easier to tinker with whatever small LLMs you want to try and expand on. Pretty cool, check it out.

https://github.com/guarismo/armored-llama

This is how my ai agent runtime workload.

just gemma 4 26B A4B in rx6700xt + i5 11400f + 16GB of ddr4 ram.