Model guides
What to deploy, and how
Memory per quantization and the sampling settings we recommend when starting an instance.
| Model | Params | Context | Smallest quant | Temp | License |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flash DeepSeekfeatured | 552B MoE (8B to 16B active) | 1M | UD-IQ1_S 233.0 GB | 1.0 | MIT |
| Gemma 4 31B Googlefeatured | 31B dense | 262K | Q4_K_M 18.3 GB | 1.0 | Apache 2.0 |
| Qwen3.8 27B Alibaba Qwenfeatured | 27B dense | 262K | UD-Q4_K_XL 17.6 GB | 1.0 | Apache 2.0 |
| Gemma 4 26B A4B Google | 26B MoE (3.8B active) | 262K | MXFP4 16.6 GB | 1.0 | Apache 2.0 |
| GLM-5.3 Z.ai | 753B MoE | 1M | UD-IQ1_S 216.7 GB | 1.0 | GLM-5.3 License |
| gpt-oss 120B OpenAI | 117B MoE (5.1B active) | 131K | MXFP4 63.4 GB | 1.0 | Apache 2.0 |
| Nemotron 3.5 Lightning 30B A3B NVIDIA | 30B hybrid MoE (3B active) | 262K | Q8_0 35.8 GB | 1.0 | OpenMDW 1.1 |
| Qwen3 Coder Next Alibaba Qwen | 80B MoE (3B active) | 262K | Q4_K_M 48.5 GB | 1.0 | Apache 2.0 |
| Qwen3.8 Flash Next Alibaba Qwen | 125B MoE (6B active) | 262K | UD-Q2_K_XL 79.6 GB | 1.0 | Qwen Community License 1.0 |