The Mac Studio M5 Ultra 256GB has 256 GB of unified memory and 1200 GB/s of memory bandwidth, which is what mostly decides generation speed. Below is the expected speed for each model at the best quantization that fits.
| Model | Quantization | Memory | Generation | First token |
|---|---|---|---|---|
| Qwen 3.6/3.8 27B | Q4_K_M | Fits | 47 tok/s | 0.4 s |
| Qwen 3.8 Flash-Next | Q4_K_M | Fits | 161 tok/s | 0.3 s |
| GLM-5.3 Flash | Q4_K | Fits | 63 tok/s | 0.3 s |
| Gemma 4 26B A4B | Q4_K_M | Fits | 257 tok/s | 0.3 s |
| GPT-OSS 20B | MXFP4 | Fits | 307 tok/s | 0.3 s |
| Qwen 3.5/3.6 35B A3B | Q4_K_M | Fits | 347 tok/s | 0.3 s |
| Gemma 4 31B | Q4_K_M | Fits | 35 tok/s | 0.4 s |
| Qwen 3.5 9B | Q4_K_M | Fits | 113 tok/s | 0.3 s |
| DeepSeek V4 Flash | UD-Q4_K_XL | Fits | 97 tok/s | 0.3 s |
| DeepSeek V4.1 Flash | Q2_K | Doesn't fit | Won't run | — |
| GPT-OSS 120B | MXFP4 | Fits | 111 tok/s | 0.3 s |
| Llama 3.1 8B | Q4_K_M | Fits | 130 tok/s | 0.3 s |
| Gemma 4 12B | Q4_K_M | Fits | 90 tok/s | 0.3 s |
| GLM-5.3 | UD-IQ1_S | Doesn't fit | Won't run | — |
| Nemotron 3.5 Lightning 30B A3B | Q4_K_M | Fits | 267 tok/s | 0.3 s |
Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.