Qwen3.8-Flash-Next-GSQ-RCO (IQ3_S): ~20-30 tok/sec decode & 300-90k tok/sec prefill on 12GB VRAM + 32GB RAM + NVME
1.95T2 sourcer/LocalLLaMA
r/LocalLLaMAoriginal source ↗
Source record
Published by r/LocalLLaMA (T2 source). The original is at https://www.reddit.com/r/LocalLLaMA/comments/1x1mzsg/qwen38flashnextgsqrco_iq3_s_2030_toksec_decode/.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryReddit user reports running the Qwen3.8-Flash-Next-GSQ-RCO model quantized to IQ3_S on a 12GB VRAM + 32GB RAM + NVME setup, achieving roughly 20-30 tok/sec decode and 300-90k tok/sec prefill.
Why it mattersUseful concrete reference for anyone fitting a quantized large model on a 12GB VRAM consumer GPU with RAM and NVME offloading.
Cited by
No citations on record.