How fast will Nemotron 3.5 Lightning 30B A3B feel on the NVIDIA DGX Spark 128GB?

Nemotron 3.5 Lightning 30B A3B on the NVIDIA DGX Spark 128GB

Nemotron 3.5 Lightning 30B A3B is a 31.58B-parameter mixture-of-experts model (3B active per token). At Q4_K_M it needs about 26.0 GB of memory. The NVIDIA DGX Spark 128GB has 119 GB usable, so it fits entirely in memory. Expect about 50 tokens/sec (24–75), with the first token after 0.3 s for a short prompt. This estimate is interpolated from benchmarks of similar setups.

Speed by quantization

QuantizationFile sizeMemoryGenerationFirst tokenSource
Q4_K_M25.3 GBFits50 tok/s0.3 sDerived
Q8_035.0 GBFits36 tok/s0.3 sDerived
FP1665.8 GBFits19 tok/s0.3 sDerived

Nemotron 3.5 Lightning 30B A3B on other hardware

Other models on the NVIDIA DGX Spark 128GB

All models on the NVIDIA DGX Spark 128GB · Nemotron 3.5 Lightning 30B A3B on all hardware

Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.