How fast will local models feel on the AMD Radeon AI PRO R9700 32GB?

Local LLM speed on the AMD Radeon AI PRO R9700 32GB

The AMD Radeon AI PRO R9700 32GB has 32 GB of memory and 644.6 GB/s of memory bandwidth, which is what mostly decides generation speed. Below is the expected speed for each model at the best quantization that fits.

ModelQuantizationMemoryGenerationFirst token
Qwen 3.6/3.8 27BQ4_K_MFits25 tok/s0.4 s
Qwen 3.8 Flash-NextQ2_KDoesn't fitWon't run—
GLM-5.3 FlashUD-IQ1_SDoesn't fitWon't run—
Gemma 4 26B A4BQ4_K_MFits166 tok/s0.3 s
GPT-OSS 20BMXFP4Fits159 tok/s0.3 s
Qwen 3.5/3.6 35B A3BQ4_K_MFits148 tok/s0.3 s
Gemma 4 31BQ4_K_MFits23 tok/s0.4 s
Qwen 3.5 9BQ4_K_MFits73 tok/s0.3 s
DeepSeek V4 FlashUD-IQ1_SDoesn't fitWon't run—
DeepSeek V4.1 FlashQ2_KDoesn't fitWon't run—
GPT-OSS 120BMXFP4Partial offload29 tok/s1.2 s
Llama 3.1 8BQ4_K_MFits84 tok/s0.3 s
Gemma 4 12BQ4_K_MFits58 tok/s0.3 s
GLM-5.3UD-IQ1_SDoesn't fitWon't run—
Nemotron 3.5 Lightning 30B A3BQ4_K_MFits173 tok/s0.3 s

Other hardware

Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.