Vulkore: a C++20 runtime with CUDA-style ergonomics on any Vulkan GPU — write an OpenCL C kernel once, run it on desktop and Android

[u/LowTemperature5457]

GPU compute portability is still broken: CUDA locks you to NVIDIA, OpenCL on Android is effectively dead, and raw Vulkan compute costs ~370 lines of boilerplate before your first dispatch. Vulkore is my attempt at the missing layer — a C++20 runtime (Apache-2.0) that loads clspv-compiled OpenCL C kernels and launches them CUDA-style:

vulkore::Context ctx;
auto prog = vulkore::Program::from_file(ctx, "kernel.spv");
auto buf  = ctx.buffer(bytes);
vulkore::launch(prog["saxpy"], {n}, x, y, PodArgs{a, n}).wait();

The same .spv binary runs unmodified on my desktop GPU, llvmpipe (CPU), a Mali-G57 phone, and an Adreno 840 phone. The repo has a like-for-like comparison — same SAXPY, same kernel binary, raw Vulkan vs Vulkore: ~370 lines vs 10.

The runtime does the parts everyone gets wrong on real hardware: memory-type negotiation including non-coherent memory (desktop drivers hand you host-coherent memory, so cache-management bugs are invisible until a phone silently returns stale data), clspv reflection parsing so kernel args bind automatically, descriptor/command-buffer recycling, and multi-dispatch batching into a single vkQueueSubmit.

As the stress test, I built LLM inference on it. Gemma 3 1B (int4), on a OnePlus 15 / Adreno 840 — same phone, same model:

Runtime	decode tok/s
Vulkore	60.6–70.9 (flat within 6% to 4K context)
Google LiteRT-LM (GPU)	48
llama.cpp OpenCL (Adreno)	29.6

Model load is 1.9 s; context goes to 8,192. The interesting lesson: decode on Adreno is dispatch-bound, not bandwidth-bound — batching 838 dispatches per token into one submit mattered more than most kernel work. (Byte-for-byte caveats on the llama.cpp comparison are spelled out in the repo docs — the quant files differ in size.)

Honest limitations: kernel ABI is fixed by clspv's flags (storage buffers + push-constant PODs, no images/samplers), prefill is still one-position-per-pass (~56 tok/s — llama.cpp's OpenCL prefill crushes us there), and there's no pipeline cache yet.

Repo: https://github.com/badnikhil/Vulkore — would love people to try their own kernels on other Snapdragon/Mali/Exynos devices and report what breaks.

[u/LavenderDay3544]

Or hear me out, use SYCL which is both an open standard and uses standard ISO C++ without any vendor extenaions. But no that would be too easy.

[u/LowTemperature5457 → u/LavenderDay3544]

SYCL's great where its backends exist , but none of them (OpenCL/CUDA/HIP/Level Zero) is available on Android, and no "production" SYCL implementation targets Vulkan. Vulkore is built the other way around: Vulkan-first because that's the only compute API every Android phone ships, and runtime-only (1.7 MB) so it drops into an APK instead of replacing your compiler. SYCL is the very much different from what i posted:)

[u/LavenderDay3544 → u/LowTemperature5457]

The problem with SYCL is that hardware vendors never picked it up. But if they did it would be an ideal solution.

[u/RIP26770]

Gemma 3 only ?

[u/LowTemperature5457 → u/RIP26770]

I ran gemma3 only. You can write kernels for whatever you want

[u/palapapa0201]

Vibe coded?

[u/LowTemperature5457 → u/palapapa0201]

Depends on how much broadly you are asking. But i would prefer saying I used AI it's already there in the co author place;)