Ornith 1.5 35B A3B is a 35.95B-parameter mixture-of-experts model (3B active per token). At Q4_K_M it needs about 22.5 GB of memory. The AMD RX 7900 XTX 24GB has 22 GB usable, so it only fits with part of the model offloaded to system RAM, which slows it down. Expect about 363 tokens/sec (138–588), with the first token after 1.2 s for a short prompt. This estimate is projected from the hardware specs; no benchmarks exist yet.
| Quantization | File size | Memory | Generation | First token | Source |
|---|---|---|---|---|---|
| Q4_K_M | 21.7 GB | Partial offload | 363 tok/s | 1.2 s | Projected |
| Q8_0 | 37.8 GB | Partial offload | 41 tok/s | 1.2 s | Projected |
| FP16 | 72.0 GB | Doesn't fit | Won't run | — | — |
All models on the AMD RX 7900 XTX 24GB · Ornith 1.5 35B A3B on all hardware
Estimates for a short prompt on a single device. Measured numbers come from community benchmarks (r/LocalLLaMA, localscore.ai and more); others are derived from similar setups or projected from specs. Open source on GitHub.