capy-flash v0.7.1: Qwen3.8-Flash-Next (125B MoE) now runs in 64 GB of RAM (234 tok/s on two 16 GB cards), and Qwen3.8-27B 3-bit fits on a single 16 GB card (174 tok/s)
2.85T2 sourcer/LocalLLaMA
r/LocalLLaMAoriginal source ↗
Source record
Published by r/LocalLLaMA (T2 source). The original is at https://www.reddit.com/r/LocalLLaMA/comments/1xciqnj/capyflash_v071_qwen38flashnext_125b_moe_now_runs/.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
Summarycapy-flash v0.7.1 runs Qwen3.8-Flash-Next, a 125B MoE model, at 234 tok/s across two 16 GB cards with 64 GB system RAM, and fits a 3-bit Qwen3.8-27B on a single 16 GB card at 174 tok/s.
Why it mattersConcrete throughput numbers for running 125B MoE and 27B dense models on consumer GPUs, useful for sizing local inference setups.
Cited by
No citations on record.