Qwen3.8 27B
The most downloaded open model on Hugging Face right now. A dense 27B with a 262K context that fits on a single 24 to 48 GB GPU at 4-bit.
Overview
Qwen3.8 27B is the dense member of the Qwen3.8 family and the base for a large share of the community fine-tunes trending on Hugging Face. It supports a thinking mode and an instruct (non-thinking) mode, and a native context window of 262,144 tokens.
Hardware fit
At UD-Q4_K_XL the weights are 17.6 GB, so a 24 GB card can serve it with a modest context. For long contexts or several parallel requests, use a 48 GB card or larger, or run Q8_0 on an 80 GB GPU.
| Quantization | Weights |
|---|---|
| UD-Q4_K_XL | 17.6 GB |
| Q8_0 | 29.0 GB |
| BF16 | 54.7 GB |
Sizes are the GGUF file sizes from unsloth/Qwen3.8-27B-GGUF. Budget extra VRAM for the KV cache, which grows with context size and parallel requests.
Deploying on Loadcore
- In the console, open Models and search for
unsloth/Qwen3.8-27B-GGUF. - Pick the quantization that fits the target server and start the download.
- Start an instance with the context size you need and GPU layers set to offload the full model.
- Point your tools at
http://your-server:9090/v1with your personal key.
Recommended settings
| Parameter | Value |
|---|---|
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 20 |
| repeat_penalty | 1.0 |
| min_p | 0.0 |
The model card gives separate settings for instruct mode: temperature 0.7, top_p 0.8, top_k 20 and presence_penalty 1.5. The values below are for thinking mode.
Source: Qwen/Qwen3.8-27B.