Gemma 4 31B VRAM requirements
fits any 24GB card, 3.9 GiB to spare · Q8_0 needs 31.9 GiB
Gemma 4 31B needs 17.6 GiB of VRAM at Q4_K_M (unsloth build) for weights plus overhead, and 18.6 GiB at 4K or 32K context: sliding-window attention saturates the KV cache at the 1024-token window, so between 4K and 32K context length changes nothing. Any 24GB card (RTX 3090 24GB or 4090 24GB) runs it with 3.9 GiB to spare. Unsloth's UD-Q5_K_XL, the densest 24GB fit, lands at 21.9 GiB, tight. 12GB cards run the 1–2-bit dynamic builds.
ollama run gemma4:31b — comfortable on any 24GB card, even at 32K (Ollama boots at 4K context — /set parameter num_ctx 32768 for the full window).Gemma 4 31B is Google's 31B dense model: 60 layers — 50 sliding-window (window 1024) plus 10 full attention (config.json layer_types). KV cache at 32K is 1.0 GiB per Google's Gemma 4 tech report (Table 3, int8 KV) — sliding windows cap 50 of 60 layers, far below the classic full-attention figure. At 8K the page shows the classic stacked upper bound instead: sliding-window saturation makes a separate 8K figure meaningless, and actual KV ≤ that value.
Data verified · sources pinned in Where every number comes from
How much VRAM at each quantization? (weights + overhead + KV @32K, batch 1)
| Quantization | VRAM needed @32K | KV @32K reported i · already in the left column | Smallest tracked fit | Note |
|---|---|---|---|---|
| UD_IQ2_XXS | 9.5 GiB | 1.0243 GiB | RTX 3060 12GB | measured · main column = the full 32K load |
| Q3_K_S | 13.8 GiB | 1.0243 GiB | RTX 4060 Ti 16GB | measured · main column = the full 32K load |
| Q3_K_M | 15.2 GiB | 1.0243 GiB | RTX 3090 24GB | measured · main column = the full 32K load |
| Q4_K_M | 18.6 GiB | 1.0243 GiB | RTX 3090 24GB | measured · main column = the full 32K load |
| Q5_K_M | 21.7 GiB | 1.0243 GiB | RTX 3090 24GB | measured · main column = the full 32K load |
| UD_Q5_K_XL | 21.9 GiB | 1.0243 GiB | RTX 3090 24GB | measured · main column = the full 32K load |
| UD_Q6_K_XL | 27.2 GiB | 1.0243 GiB | RTX 5090 32GB | measured · main column = the full 32K load |
| Q8_0 | 31.9 GiB | 1.0243 GiB | — | measured · main column = the full 32K load |
GB vs GiB: card names are the manufacturer's nominal GB; totals here are GiB (230 bytes) — a "24GB" card holds 24 GiB, and a file quoted 18,323,733,440 B in decimal is the same 17.1 GiB. Caliber →
The 48GB-class doors, spelled out: a dual-GPU combo (2× 24GB; not one card doubled), a workstation card (RTX 6000 Ada), or a Mac Studio (unified memory; runs, slower tok/s).
How this page computes the numbers (reproduce any cell)
- usable VRAM = nominal minus reserve, per card: 12GB → 10.5 GiB, 16GB → 14.5 GiB, 24GB → 22.5 GiB, 32GB → 30.0 GiB.
- total@32K = weights + overhead + KV@32K; KV@32K = 1.0243 GiB (config-derived — 33,564 B/token effective at 32K; hybrid models carry a constant state on top, composition in sources).
- The main column already prices the full 32K load, KV included; shorter-context figures live in the caption and FAQ where relevant.
- A build fits a card at 32K when total@32K ≤ usable; tight = under 2.5 GiB of headroom, one site-wide threshold.
Is your GPU enough? (find your card: one row tells you)
| Your GPU | VRAM | Biggest quant that fits (ceiling) | How it feels |
|---|---|---|---|
| RTX 3060 12GB | 12 GB | UD IQ2_XXS (2-bit) · 9.5 GiB | 1.0 GiB spare at 32K |
| RTX 4060 Ti 16GB | 16 GB | Q3_K_S (3-bit) · 13.8 GiB | 0.7 GiB spare at 32K |
| RTX 3090 24GB | 24 GB | UD Q5_K_XL (5-bit) · 21.9 GiB | 0.6 GiB spare at 32K |
| RTX 4090 24GB | 24 GB | UD Q5_K_XL (5-bit) · 21.9 GiB | 0.6 GiB spare at 32K |
| RTX 5090 32GB | 32 GB | UD Q6_K_XL (6-bit) · 27.2 GiB | 2.8 GiB spare at 32K |
Quants stack by size within the ladder: a card that runs a bigger build runs every smaller one, so each GPU has exactly one ceiling. Ceilings are full-GPU at 32K; sliding-window layers saturate early, so shorter context raises nothing further, and CPU offload trades tok/s for headroom.
What GPU should you buy for it? (the 24GB sweet spot logic)
| The logic | 24GB is the cheapest tier that runs Q4_K_M comfortably (18.6 GiB at 32K, 3.9 GiB to spare — sliding-window KV saturates, so context length does not change that). Below it, 12GB runs the 1–2-bit dynamic builds. Above it, a 32GB RTX 5090 reaches UD Q6_K_XL; nice, not necessary. |
| Why it fits | Sliding-window attention caps KV at 1.0 GiB from the 1024-token window — 50 of 60 layers never see past it — so the 31B dense model fits 24GB at Q4 with room to spare. |
⚠ Tight is a site-wide threshold: under 2.5 GiB of headroom. Q4_K_M @32K leaves 3.9 GiB (comfortable). The tightest fit on 24GB is unsloth's UD-Q5_K_XL: 21.9 GiB, about 0.6 GiB to spare (standard Q5_K_M: 21.7 GiB, also tight). Both totals already include the reported KV figure, so these verdicts are stable — not flip-check rows.
Tracked builds (actual file sizes, not theory)
| Build | Publisher | File size | 24GB @32K | Source |
|---|---|---|---|---|
| UD_IQ2_XXS | unsloth | 7.9 GiB | OK · 9.5 | HF @c1ac76e9 verified |
| Q3_K_S | unsloth | 12.3 GiB | OK · 13.8 | HF @c1ac76e9 verified |
| Q3_K_M | unsloth | 13.7 GiB | OK · 15.2 | HF @c1ac76e9 verified |
| Q4_K_M | unsloth | 17.1 GiB | OK · 18.6 | HF @c1ac76e9 verified |
| Q5_K_M | unsloth | 20.2 GiB | TIGHT · 21.7 | HF @c1ac76e9 verified |
| UD_Q5_K_XL | unsloth | 20.4 GiB | TIGHT · 21.9 | HF @c1ac76e9 verified |
| UD_Q6_K_XL | unsloth | 25.6 GiB | NO · 27.2 | HF @c1ac76e9 verified |
| Q8_0 | ggml-org | 30.4 GiB | NO · 31.9 | HF @4fa4fdf3 verified |
Quant studios aim their ladders at the same fixed GPU tiers this page anchors on: unsloth's UD-Q5_K_XL lands at 21.9 GiB against 22.5 GiB usable — a designed marginal fit (unsloth model card, linked, reported). Same quant, different builds: Google's own BF16 is 1.85% over unsloth's by exact file bytes (62,546,338,248 vs 61,413,190,528 B) — display rounding hides it. Where 24GB ends: UD-Q5_K_XL; Q6_K (25.0 GiB) is the first door that misses.
Self-host or API?
Why in / out? Output tokens cost more than input: a prompt is read in one parallel pass, while output is generated token-by-token — costlier to serve. For example, gemma-4-31b-it listed at $0.09 in / $0.34 out per 1M tokens in the 2026-09-18 OpenRouter snapshot; blended at a typical chat mix (4 input : 1 output) ≈ $0.14/1M.
API prices float; the crossover moves less than one decade either way — so this page answers the question that doesn't need live prices: how much monthly usage makes self-hosting worth considering at all.
| Monthly usage | Verdict | Why |
|---|---|---|
| < 80M tokens | API | Even the cheapest self-host ($30/mo) never breaks even against the API bill at this usage — don't bother calculating. |
| 80M-700M tokens | Conditional | The decisive variable is whether you already own a 24GB card — amortization dominates the self-host bill. |
| > 700M tokens | Self-host | Per-token API billing stops scaling here; consider a managed dedicated instance too — at 2B+ tokens/mo it often beats DIY. |
Robustness: at 0.09/1M blended the crossover sits near 650 tokens/mo; at 0.34/1M near 90/mo. The three bands hold under ±2.5× API-price and ±2× self-host-cost floats. Self-host cost basis: $800-1900 GPU amortized over 30 months ($27-63/mo) + ~350W × 4h/day × $0.15/kWh (≈ $6/mo). Live prices via OpenRouter (linked, not re-verified by us); the band is a 31B-class reference from the 2026-09-18 OpenRouter listing, not a pinned price for this model.
Where every number comes from (every figure carries its tag)
Totals: measured weights + reported KV (vendor tech report, Table 3) + measured overhead — labeled, additive. This page exists for one config: the 24–35B dense agent tier × Q4_K-class quant × 32K context on a 24GB card — the sweet spot where demand (the smallest models that still run agents) meets supply (quant studios' ladders, built around the same fixed GPU tiers).
| Tag | Slot | What it is | What the verdict can claim |
|---|---|---|---|
| reported | model | Hugging Face config.json (hypothesis-parameter carrier) | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | static | Hugging Face model page | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | kv | KV cache @ 32K — Gemma 4 tech report Table 3 (int8 KV, +1.10 GB @32K); @8K not separately sourced (sliding-window saturation) | Tight, conditional — stands on the linked source, re-checkable by you. |
| measured | overhead | Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as 8K row) | Determinate — "it runs" means it runs. |
| Hugging Face config.json (hypothesis-parameter carrier) | @842da379 | accessed 2026-09-06 |
| Hugging Face model page | @842da379 | accessed 2026-09-06 |
| KV cache @ 32K — Gemma 4 tech report Table 3 (int8 KV, +1.10 GB @32K); @8K not separately sourced (sliding-window saturation) | latest | accessed 2026-09-05 |
| Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as 8K row) | latest | accessed 2026-09-05 |
FAQ
Does Gemma 4 31B fit in 24GB VRAM?
Yes, comfortably at 4-bit. The unsloth Q4_K_M build totals 18.6 GiB at 32K context against 22.5 GiB usable on an RTX 3090 24GB or 4090 24GB — 3.9 GiB to spare. The densest build that fits is UD_Q5_K_XL at 21.9 GiB, tight.
How much VRAM does Gemma 4 31B need at Q4_K_M?
18.6 GiB at 4K context or 32K: 17.1 GiB weights + 0.5 GiB runtime overhead + 1.0 GiB KV cache. The KV figure is reported (Google's Gemma 4 tech report, Table 3, int8 KV caching), and sliding-window saturation makes it identical at 4K and 32K: 50 of 60 layers never see past the 1024-token window.
What is the smallest GPU that runs Gemma 4 31B?
An RTX 3060 12GB tops out at UD_IQ2_XXS (9.5 GiB at 32K, tight), and a 16GB card reaches Q3_K_S (13.8 GiB, tight). Q4_K_M, the standard 4-bit build, needs a 24GB card.
Why is the KV cache about 1.0 GiB instead of the classic 5.78 GiB estimate?
50 of its 60 layers are sliding-window attention (window 1024), so the cache saturates at the window instead of growing with context. Measured int8 KV at 32K is 1.0 GiB per the Gemma 4 tech report. Sliding-window saturation also means shorter context shrinks nothing further: the quoted KV is one reported figure, not per-context math.
Can Gemma 4 31B run above Q5 on a single consumer GPU?
Not on a 24GB card: Q6_K totals 25.0 GiB at 32K, 2.5 GiB over the 22.5 GiB usable. An RTX 5090 32GB reaches UD_Q6_K_XL at 27.2 GiB. Q8_0 (31.9 GiB) needs a 48GB-class card, and BF16 at 59.8 GiB needs 80GB-class hardware.
UD-Q5_K_XL also fits my 24GB card — why does this page still recommend Q4_K_M?
Because Q4_K_M is the community default: it matches what most people actually download, keeps 3.9 GiB of headroom for longer context and everyday workloads, and the quality step from Q4 to Q5 is small next to what extra context buys an agent. UD-Q5_K_XL is a legitimate choice if you want the densest 24GB fit — listed here as a display fact (21.9 GiB, 0.6 GiB to spare, tight), not the recommendation. Quality-critical and short-context? Take the Q5. Most setups: Q4 plus headroom is the steadier default.
How does this site compute these VRAM numbers?
Weights are measured file sizes at pinned revisions (sources below); totals add runtime overhead plus the KV cache at 32K context. Here KV is a reported 1.02 GiB (int8, tech report Table 3) — and it does not shrink at shorter contexts (sliding-window saturation), which is why this page prices the full 32K load only. A build fits a card at 32K when the total clears usable VRAM (nominal minus reserve), and tight means under 2.5 GiB of headroom — the full formula lives in the method box under the quantization table.