Splash on a 40-core M5 Max: +20% decode by tuning the kernels for your own chip
2.70T2 sourcer/LocalLLaMA
r/LocalLLaMAoriginal source ↗
Source record
Published by r/LocalLLaMA (T2 source). The original is at https://www.reddit.com/r/LocalLLaMA/comments/1wpw0jn/splash_on_a_40core_m5_max_20_decode_by_tuning_the/.
Pipeline notes
The summary and note below are generated by the signal pipeline — they are Beyond Desk’s reading, not quotations from the source.
SummaryA LocalLLaMA user reports a 20% decode speedup on a 40-core Apple M5 Max by tuning inference kernels specifically for that chip using a tool called Splash, rather than relying on generic kernel defaults.
Why it mattersFirsthand benchmark on a new Apple Silicon part showing real payoff from hardware-specific kernel tuning. Useful reference for anyone running local LLMs on M-series chips.
Cited by
No citations on record.