How fast will local models feel on the NVIDIA DGX Spark 128GB?

Local LLM speed on the NVIDIA DGX Spark 128GB

The NVIDIA DGX Spark 128GB has 128 GB of memory and 273 GB/s of memory bandwidth, which is what mostly decides generation speed. Below is the expected speed for each model at the best quantization that fits.

Model	Quantization	Memory	Generation	First token
Qwen 3.6/3.8 27B	Q4_K_M	Fits	6.5 tok/s	0.3 s
GLM-5.3 Flash	UD-Q2_K_XL	Fits	18 tok/s	0.3 s
GPT-OSS 20B	MXFP4	Fits	53 tok/s	0.3 s
Gemma 4 26B A4B	Q4_K_M	Fits	44 tok/s	0.3 s
Qwen 3.5/3.6 35B A3B	Q4_K_M	Fits	60 tok/s	0.3 s
Ornith 1.5 35B A3B	Q4_K_M	Fits	47 tok/s	0.3 s
DeepSeek V4 Flash	UD-IQ3_S	Fits	22 tok/s	0.3 s
GPT-OSS 120B	MXFP4	Fits	40 tok/s	0.3 s
Gemma 4 31B	Q4_K_M	Fits	6.0 tok/s	0.3 s
Llama 3.1 8B	Q4_K_M	Fits	35 tok/s	0.3 s
Qwen 3.5 9B	Q4_K_M	Fits	15 tok/s	0.3 s
DeepSeek R1 Distill 14B	Q4_K_M	Fits	19 tok/s	0.3 s
Ornith 1.5 9B	Q4_K_M	Fits	15 tok/s	0.3 s
Muse Glimmer 30B	Q4_K_M	Fits	4.3 tok/s	0.3 s
Qwen3 30B A3B	Q4_K_M	Fits	42 tok/s	0.3 s

Other hardware

Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.