Gemma 4 31B

Overview

Gemma 4 31B is the dense flagship of Google's Gemma 4 family, instruction tuned and able to take image input. It has a 262,144 token context window.

Hardware fit

Q4_K_M is 18.3 GB and Q8_0 is 33.2 GB. A 48 GB card such as an L40S runs Q8_0 with room for context, and a 24 GB card runs Q4_K_M with a shorter context. Vision input needs the separate mmproj file from the GGUF repo.

Quantization	Weights
Q4_K_M	18.3 GB
Q6_K	25.2 GB
Q8_0	33.2 GB

Sizes are the GGUF file sizes from unsloth/gemma-4-31B-it-GGUF. Budget extra VRAM for the KV cache, which grows with context size and parallel requests.

Deploying on Loadcore

1. In the console, open Models and search for unsloth/gemma-4-31B-it-GGUF.
2. Pick the quantization that fits the target server and start the download.
3. Start an instance with the context size you need and GPU layers set to offload the full model.
4. Point your tools at http://your-server:9090/v1 with your personal key.

Recommended settings

Parameter	Value
temperature	1.0
top_p	0.95
top_k	64
repeat_penalty	1.0
min_p	0.0

Values below are Google's recommended defaults from the model card.

Source: google/gemma-4-31B-it.