How fast will GPT-OSS 20B feel on the AMD Instinct MI50 32GB?

GPT-OSS 20B on the AMD Instinct MI50 32GB

GPT-OSS 20B is a 20.91B-parameter mixture-of-experts model (3.6B active per token). At MXFP4 it needs about 12.9 GB of memory. The AMD Instinct MI50 32GB has 30 GB usable, so it fits entirely in memory. Expect about 316 tokens/sec (158–474), with the first token after 0.4 s for a short prompt. This estimate is interpolated from benchmarks of similar setups.

Speed by quantization

Quantization	File size	Memory	Generation	First token	Source
MXFP4	12.1 GB	Fits	316 tok/s	0.4 s	Derived

GPT-OSS 20B on other hardware

H100 80GB (cloud) · 700 tok/s at MXFP4
NVIDIA RTX 5090 32GB · 375 tok/s at MXFP4
NVIDIA RTX PRO 6000 96GB · 375 tok/s at MXFP4
Mac Studio M5 Ultra 256GB · 307 tok/s at MXFP4
H200 141GB (cloud) · 301 tok/s at MXFP4
AMD RX 7900 XTX 24GB · 296 tok/s at MXFP4
Mac Studio M5 Ultra 512GB · 277 tok/s at MXFP4
NVIDIA RTX 4090 24GB · 211 tok/s at MXFP4
Mac Studio M3 Ultra 512GB · 210 tok/s at MXFP4
NVIDIA RTX 5080 16GB · 201 tok/s at MXFP4
NVIDIA RTX 3090 24GB · 196 tok/s at MXFP4
NVIDIA Tesla V100 32GB · 188 tok/s at MXFP4
NVIDIA RTX 5070 Ti 16GB · 187 tok/s at MXFP4
AMD Radeon AI PRO R9700 32GB · 159 tok/s at MXFP4
Mac Studio M4 Max 128GB · 140 tok/s at MXFP4
MacBook Pro M5 Max 128GB · 139 tok/s at MXFP4
AMD RX 9070 XT 16GB · 136 tok/s at MXFP4
AMD RX 7900 XT 20GB · 123 tok/s at MXFP4
NVIDIA RTX 5060 Ti 16GB · 94 tok/s at MXFP4
AMD Ryzen AI Max+ PRO 495 192GB (Gorgon Halo) · 84 tok/s at MXFP4
MacBook Pro M5 Pro 48GB · 82 tok/s at MXFP4
Mac mini M5 Pro 64GB · 79 tok/s at MXFP4
Intel Arc Pro B70 32GB · 76 tok/s at MXFP4
AMD Ryzen AI Max+ 395 128GB (Strix Halo) · 74 tok/s at MXFP4
MacBook M4 Pro 24GB · 70 tok/s at MXFP4
NVIDIA RTX Spark N1X 128GB (laptop) · 60 tok/s at MXFP4
NVIDIA RTX 3060 12GB · 58 tok/s at MXFP4
NVIDIA DGX Spark 128GB · 57 tok/s at MXFP4
CPU-only (Ryzen 9 9950X, 96GB DDR5) · 29 tok/s at MXFP4

Other models on the AMD Instinct MI50 32GB

Qwen 3.6/3.8 27B · 25 tok/s at Q4_K_M
Qwen 3.8 Flash-Next · won't run
GLM-5.3 Flash · won't run
Gemma 4 26B A4B · 56 tok/s at Q4_K_M
Qwen 3.5/3.6 35B A3B · 356 tok/s at Q4_K_M
Gemma 4 31B · 36 tok/s at Q4_K_M
Qwen 3.5 9B · 116 tok/s at Q4_K_M
DeepSeek V4 Flash · won't run
DeepSeek V4.1 Flash · won't run
GPT-OSS 120B · won't run
Llama 3.1 8B · 72 tok/s at Q4_K_M
Gemma 4 12B · 92 tok/s at Q4_K_M
GLM-5.3 · won't run
Nemotron 3.5 Lightning 30B A3B · 274 tok/s at Q4_K_M

All models on the AMD Instinct MI50 32GB · GPT-OSS 20B on all hardware

Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.