The CPU-only (Ryzen 9 9950X, 96GB DDR5) has 96 GB of memory and 89.6 GB/s of memory bandwidth, which is what mostly decides generation speed. Below is the expected speed for each model at the best quantization that fits.
| Model | Quantization | Memory | Generation | First token |
|---|---|---|---|---|
| Qwen 3.6/3.8 27B | Q4_K_M | Fits | 3.6 tok/s | 0.9 s |
| GLM-5.3 Flash | UD-IQ1_S | Partial offload | 14 tok/s | 2.0 s |
| GPT-OSS 20B | MXFP4 | Fits | 29 tok/s | 0.4 s |
| Gemma 4 26B A4B | Q4_K_M | Fits | 24 tok/s | 0.4 s |
| Qwen 3.5/3.6 35B A3B | Q4_K_M | Fits | 33 tok/s | 0.4 s |
| Ornith 1.5 35B A3B | Q4_K_M | Fits | 33 tok/s | 0.4 s |
| DeepSeek V4 Flash | UD-IQ1_M | Fits | 15 tok/s | 0.6 s |
| GPT-OSS 120B | MXFP4 | Fits | 22 tok/s | 0.4 s |
| Gemma 4 31B | Q4_K_M | Fits | 3.3 tok/s | 1.0 s |
| Llama 3.1 8B | Q4_K_M | Fits | 10 tok/s | 0.5 s |
| Qwen 3.5 9B | Q4_K_M | Fits | 10 tok/s | 0.5 s |
| DeepSeek R1 Distill 14B | Q4_K_M | Fits | 6.6 tok/s | 0.6 s |
| Ornith 1.5 9B | Q4_K_M | Fits | 10 tok/s | 0.5 s |
| Muse Glimmer 30B | Q4_K_M | Fits | 3.0 tok/s | 1.0 s |
| Qwen3 30B A3B | Q4_K_M | Fits | 30 tok/s | 0.4 s |
Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.