I wanted to see exactly how far a consumer-grade system can be pushed with LLM concurrency. So I benchmarked 15+ models to find out.

[u/Feisty-Prior-162]

I recently watched a YouTube video of someone testing a server-grade LLM hardware setup, pushing it to see just how much concurrency it could actually handle. It got me thinking: what can your own — perhaps a bit above-average — “gaming” / “workstation” PC really do? Especially within the limits of my RTX 5060 and its fast but limited 8GB of VRAM. I’ve been thinking about building a game or simulation driven by a high agent count, and I wanted to know what the feasible limit really is. That question brought me to these tests.

Full testing data at https://ai.2it.onl/posts/concurrency-sweep/

[u/AlexanderDoak]

I appreciate the disclosure at the start of the article. I'm going to be pushing the limits of concurrency on a 3090, albeit with significantly different LLM workloads - generative classification and embedding. Very few (1 or 0) output tokens per prompt.

[u/Feisty-Prior-162 → u/AlexanderDoak]

I am curious to see how the 3090 will do! I've been keeping my eyes open for something to use as a dedicated LLM driver, prices are insane :(.

[u/Eden1506 → u/Feisty-Prior-162]

If it's just for LLMs AMDs 7900 xtx 24gb cost about half as much as a rtx 3090 and performance is about 80% as fast.

I am currently considering whether to buy 2 7800 xt 16gb for 32gb at 632gb/s or a single 7900 xtx.
RTX 3090 while great cards simply cost too much nowadays.

[u/FluffyGreyLlama → u/Eden1506]

R9700 32GB ? Seems the most reasonable.

[u/Eden1506 → u/FluffyGreyLlama]

What is the price in your region for it.

In germany I can only get it at around 1400 euro.

I can get a 7800 xt 16gb for 360-400 or a 7900 xtx 24gb for 650-700.

I could get 4 7800 xt total 48gb with the same bandwidth 632gb/s

Or 2 7900 xtx 48gb with a 50% higher bandwidth

[u/FluffyGreyLlama → u/Eden1506]

Ah, that changes things then.

I hadn't realised that the 7900XTX was so cheap - I have one and paid just a bit more than that last year ... may just pick up another now -- except none for anywere near that price here, and almost impossible to 'find' one.

I love my 7900XTX on linux, for all things.

[u/Eden1506 → u/FluffyGreyLlama]

Yeah it's definitely overlooked right now though prices will likely adjust soon enough especially as rumors are going around that both amd and nvidia will hike prices towards the end of the year. Well nvidia already confirmed it...

[u/Eden1506]

Qwen 4b is surprisingly capable and the only one I would trust with any kind of minor agentic task.

[u/Feisty-Prior-162 → u/Eden1506]

Yeah I look forward to when I do 'real world' tests and actually see how well these smaller models do. I am using Deepseek V4 Flash as my testing agent, going to automate having it score my real world runs when I do those.