How fast will Qwen 3.5 9B feel on the AMD RX 9070 XT 16GB?

Qwen 3.5 9B on the AMD RX 9070 XT 16GB

Qwen 3.5 9B is a 9.65B-parameter model. At Q4_K_M it needs about 6.4 GB of memory. The AMD RX 9070 XT 16GB has 15 GB usable, so it fits entirely in memory. Expect about 58 tokens/sec (51–66), with the first token after 0.3 s for a short prompt. This estimate is based on community benchmarks of this exact setup.

Speed by quantization

QuantizationFile sizeMemoryGenerationFirst tokenSource
Q4_K_M5.7 GBFits58 tok/s0.3 sMeasured
Q8_09.5 GBFits57 tok/s0.3 sMeasured
FP1617.9 GBPartial offload9.1 tok/s1.3 sProjected

Qwen 3.5 9B on other hardware

Other models on the AMD RX 9070 XT 16GB

All models on the AMD RX 9070 XT 16GB · Qwen 3.5 9B on all hardware

Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.