How fast will local models feel on the NVIDIA RTX 5060 Ti 16GB?

Local LLM speed on the NVIDIA RTX 5060 Ti 16GB

The NVIDIA RTX 5060 Ti 16GB has 16 GB of memory and 448 GB/s of memory bandwidth, which is what mostly decides generation speed. Below is the expected speed for each model at the best quantization that fits.

Model	Quantization	Memory	Generation	First token
Qwen 3.6/3.8 27B	Q4_K_M	Partial offload	8.2 tok/s	1.4 s
GLM-5.3 Flash	UD-IQ1_S	Doesn't fit	Won't run	—
GPT-OSS 20B	MXFP4	Fits	87 tok/s	0.3 s
Gemma 4 26B A4B	Q4_K_M	Partial offload	55 tok/s	1.2 s
Qwen 3.5/3.6 35B A3B	Q4_K_M	Partial offload	58 tok/s	1.2 s
Ornith 1.5 35B A3B	Q4_K_M	Partial offload	60 tok/s	1.2 s
DeepSeek V4 Flash	UD-IQ1_S	Doesn't fit	Won't run	—
GPT-OSS 120B	MXFP4	Doesn't fit	Won't run	—
Gemma 4 31B	Q4_K_M	Partial offload	6.8 tok/s	1.4 s
Llama 3.1 8B	Q4_K_M	Fits	57 tok/s	0.3 s
Qwen 3.5 9B	Q4_K_M	Fits	25 tok/s	0.3 s
DeepSeek R1 Distill 14B	Q4_K_M	Fits	29 tok/s	0.3 s
Ornith 1.5 9B	Q4_K_M	Fits	24 tok/s	0.3 s
Muse Glimmer 30B	Q4_K_M	Partial offload	5.9 tok/s	1.4 s
Qwen3 30B A3B	Q4_K_M	Partial offload	62 tok/s	1.2 s

Other hardware

Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.