I built an RK3588 NPU runtime without any proprietary inference libraries: Qwen3.5-4B at ~10 tok/s, ~2x faster than vendor SDK.
2.85T2 sourcer/LocalLLaMA
r/LocalLLaMAoriginal source ↗
Source record
Published by r/LocalLLaMA (T2 source). The original is at https://www.reddit.com/r/LocalLLaMA/comments/1xb45bq/i_built_an_rk3588_npu_runtime_without_any/.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryDeveloper built a custom runtime for the RK3588 NPU without proprietary inference libraries, reporting ~10 tok/s with Qwen3.5-4B, roughly 2x faster than the vendor SDK on the same hardware.
Why it mattersConcrete engineering result on edge NPU inference with reproducible numbers; relevant to anyone deploying local models on RK3588 hardware without vendor lock-in.
Cited by
No citations on record.