Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s
3.00T2 sourceHacker News · Show HN (50+ points)
Source record
Published by Hacker News · Show HN (50+ points) (T2 source). The original is at https://github.com/carloslfu/slotstream.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryA developer built 'slotstream,' a tool that runs the 125B-parameter Qwen3.8-Flash-Next in 4-bit on Macs with as little as 16GB RAM using expert-offloading and SSD streaming on MLX/Swift, achieving ~12 tok/s on a 48GB Mac. Ships with auto memory/speed mode; MTP speculative decoding planned.
Why it mattersFirsthand tool with working install and measured throughput, showing a concrete technique for fitting a 125B model onto consumer Mac hardware via SSD streaming.
Cited by
No citations on record.