How fast will local models feel on the NVIDIA RTX 5070 Ti 16GB?

Local LLM speed on the NVIDIA RTX 5070 Ti 16GB

The NVIDIA RTX 5070 Ti 16GB has 16 GB of memory and 896 GB/s of memory bandwidth, which is what mostly decides generation speed. Below is the expected speed for each model at the best quantization that fits.

ModelQuantizationMemoryGenerationFirst token
Qwen 3.6/3.8 27BQ4_K_MPartial offload11 tok/s1.3 s
Qwen 3.8 Flash-NextQ2_KDoesn't fitWon't run—
GLM-5.3 FlashUD-IQ1_SDoesn't fitWon't run—
Gemma 4 26B A4BQ4_K_MPartial offload74 tok/s1.2 s
GPT-OSS 20BMXFP4Fits187 tok/s0.3 s
Qwen 3.5/3.6 35B A3BQ4_K_MPartial offload61 tok/s1.2 s
Gemma 4 31BQ4_K_MPartial offload8.3 tok/s1.3 s
Qwen 3.5 9BQ4_K_MFits69 tok/s0.3 s
DeepSeek V4 FlashUD-IQ1_SDoesn't fitWon't run—
DeepSeek V4.1 FlashQ2_KDoesn't fitWon't run—
GPT-OSS 120BMXFP4Doesn't fitWon't run—
Llama 3.1 8BQ4_K_MFits50 tok/s0.3 s
Gemma 4 12BQ4_K_MFits55 tok/s0.3 s
GLM-5.3UD-IQ1_SDoesn't fitWon't run—
Nemotron 3.5 Lightning 30B A3BQ4_K_MPartial offload40 tok/s1.2 s

Other hardware

Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.