Qwen3.6-27B VRAM requirements
fits any 24GB card, 4.3 GiB to spare · community AWQ tops out at 21.5 GiB, 1.0 to spare (tight)
Qwen3.6-27B needs 16.7 GiB at Q4_K_M (unsloth build) at 8K context and 18.2 GiB at 32K (reported KV counted) — any 24GB card runs it with 4.3 GiB to spare. Q4_K_M is the community default. The densest 24GB fit is community AWQ-INT4 at 21.5 GiB (tight, vLLM-format); the densest GGUF build is unsloth's UD-Q5_K_XL at 21.2 GiB.
ollama run qwen3.6:27b — comfortable on any 24GB card, even at 32K (Ollama boots at 4K context — /set parameter num_ctx 32768 for the full window).Qwen3.6-27B is Alibaba's hybrid-linear (GDN) model: 64 layers — 48 linear-attention plus 16 full attention (config.json layer_types), a 256K native context window. KV cache at 32K is 2.0 GiB per kaitchup's measurement — 17.2 GB bf16 for the full-attention layers at the native 262,144-token context, scaled to 32K. That is growing KV only: the GDN layers' constant recurrent state is not published (~0.07 GiB uncounted, so totals slightly understate).
Data verified · sources pinned in Where every number comes from
How much VRAM at each quantization? (weights + overhead + KV @32K, batch 1)
| Quantization | VRAM needed @32K | KV @32K reported i · already in the left column | Smallest tracked fit | Note |
|---|---|---|---|---|
| UD_IQ2_XXS | 11.2 GiB | 2 GiB | RTX 4060 Ti 16GB | measured · main column = the full 32K load |
| Q3_K_S | 14.0 GiB | 2 GiB | RTX 4060 Ti 16GB | measured · main column = the full 32K load |
| Q4_K_M | 18.2 GiB | 2 GiB | RTX 3090 24GB | measured · main column = the full 32K load |
| AWQ | 21.5 GiB | 2 GiB | RTX 3090 24GB | measured · main column = the full 32K load |
| Q8_0 | 29.1 GiB | 2 GiB | RTX 5090 32GB | measured · main column = the full 32K load |
GB vs GiB: card names are the manufacturer's nominal GB; totals here are GiB (230 bytes) — a "24GB" card holds 24 GiB, and a file quoted 16,817,244,384 B in decimal is the same 15.7 GiB. Caliber →
The 48GB-class doors, spelled out: a dual-GPU combo (2× 24GB; not one card doubled), a workstation card (RTX 6000 Ada), or a Mac Studio (unified memory; runs, slower tok/s).
How this page computes the numbers (reproduce any cell)
- usable VRAM = nominal minus reserve, per card: 12GB → 10.5 GiB, 16GB → 14.5 GiB, 24GB → 22.5 GiB, 32GB → 30.0 GiB.
- total@32K = weights + overhead + KV@32K; KV@32K = 2 GiB (config-derived — 65,536 B/token effective at 32K; hybrid models carry a constant state on top, composition in sources).
- The main column already prices the full 32K load, KV included; shorter-context figures live in the caption and FAQ where relevant.
- A build fits a card at 32K when total@32K ≤ usable; tight = under 2.5 GiB of headroom, one site-wide threshold.
Is your GPU enough? (find your card: one row tells you)
| Your GPU | VRAM | Biggest quant that fits (ceiling) | How it feels |
|---|---|---|---|
| RTX 3060 12GB | 12 GB | — | — |
| RTX 4060 Ti 16GB | 16 GB | Q3_K_S (3-bit) · 14.0 GiB | 0.5 GiB spare at 32K |
| RTX 3090 24GB | 24 GB | AWQ (4-bit) · 21.5 GiB | 1.0 GiB spare at 32K |
| RTX 4090 24GB | 24 GB | AWQ (4-bit) · 21.5 GiB | 1.0 GiB spare at 32K |
| RTX 5090 32GB | 32 GB | Q8_0 (8-bit) · 29.1 GiB | 0.9 GiB spare at 32K |
Quants stack by size: each GPU has exactly one ceiling — a card that runs a bigger build runs every smaller one. Main-column totals price the 32K load (weights + overhead + reported KV @32K); shorter context lowers every row by most of the KV term, and CPU offload trades tok/s for headroom.
What GPU should you buy for it? (the 24GB sweet spot logic)
| The logic | 24GB is the cheapest tier that runs Q4_K_M with real headroom: 18.2 GiB at 32K with the reported KV counted, 4.3 GiB to spare. Below it, 16GB tops out at Q3_K_S (0.5 spare) and 12GB is 8K-only territory (smallest build 9.7 GiB at 8K). Above it, an RTX 5090 32GB reaches Q8_0; nice, not necessary. |
| Why it fits | 48 of the 64 layers are GDN linear attention holding a constant state; at 32K the full-attention KV rides a 2.0 GiB reported figure — so the 27B hybrid fits 24GB at Q4 with room to spare. |
⚠ Tight is a site-wide threshold: under 2.5 GiB of headroom against usable VRAM. Q4_K_M @32K leaves 4.3 GiB on a 24GB card — comfortable. The tight fits: community AWQ-INT4 totals 21.5 GiB (1.0 to spare), UD-Q5_K_XL 21.2 GiB (1.3 to spare). Just past the line: Q6_K totals 23.5 GiB at 32K — 1.0 GiB over the 22.5 GiB usable; it fits 24GB at 8K context (22.0 GiB), and the GPU ladder above prices that.
Tracked builds (actual file sizes, not theory)
| Build | Publisher | File size | 24GB @32K | Source |
|---|---|---|---|---|
| UD_IQ2_XXS | unsloth | 8.7 GiB | OK · 11.2 | HF @82d411ac verified |
| Q3_K_S | unsloth | 11.5 GiB | OK · 14.0 | HF @82d411ac verified |
| Q4_K_M | unsloth | 15.7 GiB | OK · 18.2 | HF @82d411ac verified |
| Q4_K_M (ggml-org) | ggml-org | 17.8 GiB | TIGHT · 20.3 | HF @8a7ee08e verified |
| AWQ | cyankiwi | 19.0 GiB | TIGHT · 21.5 | HF @e5cc0400 verified |
| Q8_0 | ggml-org | 26.6 GiB | NO · 29.1 | HF @8a7ee08e verified |
Same quant, different builds: ggml-org's Q4_K_M measures 17.8 GiB (19,095,766,304 bytes, pinned rev 8a7ee08e) — 13.4% over unsloth's 15.7 GiB (16,817,244,384 bytes, rev 82d411ac) — and the two sit side by side above. Q8_0 is the mirror case: unsloth and ggml-org ship the same bytes (28,595,76x,xxx B), so one row stands for both. The AWQ row is a community vLLM-format build (cyankiwi, pinned rev e5cc0400) — measured from its file listing, but not a llama.cpp file. Where 24GB ends at 32K: AWQ-INT4 at 21.5 GiB; Q6_K (23.5 GiB) is the first door that misses.
Self-host or API?
Why in / out? Output tokens cost more than input: a prompt is read in one parallel pass, while output is generated token-by-token — costlier to serve. For example, qwen3.6-27b listed at $0.30 in / $2.00 out per 1M tokens in the 2026-09-18 OpenRouter snapshot; blended at a typical chat mix (4 input : 1 output) ≈ $0.64/1M.
API prices float; the crossover moves less than one decade either way — so this page answers the question that doesn't need live prices: how much monthly usage makes self-hosting worth considering at all.
| Monthly usage | Verdict | Why |
|---|---|---|
| < 40M tokens | API | Even the cheapest self-host ($30/mo) never breaks even against the API bill at this usage — don't bother calculating. |
| 40M-150M tokens | Conditional | The decisive variable is whether you already own a 24GB card — amortization dominates the self-host bill. |
| > 150M tokens | Self-host | Per-token API billing stops scaling here; consider a managed dedicated instance too — at 500M+ tokens/mo it often beats DIY. |
Robustness: at 0.45/1M blended the crossover sits near 130 tokens/mo; at 0.70/1M near 40/mo. The three bands hold under ±2.5× API-price and ±2× self-host-cost floats. Self-host cost basis: $800-1900 GPU amortized over 30 months ($27-63/mo) + ~350W × 4h/day × $0.15/kWh (≈ $6/mo). Live prices via OpenRouter (linked, not re-verified by us); the band is a 27B-class reference from the 2026-09-18 OpenRouter listing, not a pinned price for this model.
Where every number comes from (every figure carries its tag)
Totals: measured weights + reported KV (community measurement, linked) + measured overhead — labeled, additive. This page exists for one config: the ~27B dense agent tier × Q4_K-class quant × 32K context on a 24GB card — the sweet spot where demand (the smallest models that still run agents) meets supply (quant studios designing marginal fits around the same fixed GPU tiers).
| Tag | Slot | What it is | What the verdict can claim |
|---|---|---|---|
| reported | model | Hugging Face config.json (hypothesis-parameter carrier) | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | static | Hugging Face model page | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | kv | KV cache @ 32K — kaitchup measured writeup: 17.2 GB bf16 for the 16 full-attention layers at the native 262,144-token context, scaled to 2.0 GiB @32K (growing KV only; flat GDN state not published, ~0.07 GiB uncounted) | Tight, conditional — stands on the linked source, re-checkable by you. |
| measured | overhead | Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | Determinate — "it runs" means it runs. |
| Hugging Face config.json (hypothesis-parameter carrier) | @6a9e13bd | accessed 2026-09-14 |
| Hugging Face model page | @6a9e13bd | accessed 2026-09-14 |
| KV cache @ 32K — kaitchup measured writeup: 17.2 GB bf16 for the 16 full-attention layers at the native 262,144-token context, scaled to 2.0 GiB @32K (growing KV only; flat GDN state not published, ~0.07 GiB uncounted) | latest | accessed 2026-09-06 |
| Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | latest | accessed 2026-09-05 |
FAQ
Does Qwen3.6-27B fit in 24GB VRAM?
Yes. The unsloth Q4_K_M build totals 18.2 GiB at 32K context against 22.5 GiB usable on an RTX 3090 24GB or 4090 24GB — 4.3 GiB to spare. The densest build that fits is community AWQ-INT4 at 21.5 GiB (1.0 GiB to spare, tight, vLLM-format); the densest GGUF build is UD-Q5_K_XL at 21.2 GiB.
How much VRAM does Qwen3.6-27B need at Q4_K_M?
18.2 GiB at 32K context: 15.7 GiB weights + 0.5 GiB runtime overhead + 2.0 GiB KV cache. The KV figure is reported (kaitchup measured 17.2 GB bf16 for the full-attention layers at the native 262,144-token context, scaled to 32K; growing KV only, the GDN constant state is not published). At 4K context the KV term shrinks eightfold to ≈0.25 GiB, so the same build runs in ≈16.4 GiB. The short form (ollama run qwen3.6:27b) pulls the ollama Q4_K_M build.
What is the smallest GPU that runs Qwen3.6-27B?
At 32K context an RTX 3060 12GB tops out at nothing — the smallest build totals 11.2 GiB against 10.5 GiB usable. At 8K context that build (UD_IQ2_XXS, 9.7 GiB) fits, tight. A 16GB card reaches Q3_K_S (14.0 GiB at 32K, 0.5 to spare). Q4_K_M, the standard 4-bit build, needs a 24GB card.
Why is the KV cache 2.0 GiB and not the classic full-attention estimate?
Because 48 of the 64 layers are GDN linear attention and hold a constant recurrent state instead of growing KV. The 2.0 GiB @32K is a measurement for the 16 full-attention layers only (kaitchup, linked): 17.2 GB bf16 at the model's native 262,144-token context, scaled down to 32K. The constant state of the linear layers is not published — roughly 0.07 GiB by config arithmetic, uncounted, so totals slightly understate. A classic all-layers bound would overstate what the 16 layers actually hold.
Can Qwen3.6-27B run above Q5 on a single consumer GPU?
Not at 32K on 24GB: Q6_K totals 23.5 GiB, 1.0 GiB over the 22.5 GiB usable — at 8K context it fits (22.0 GiB), and the GPU ladder prices that. An RTX 5090 32GB reaches Q8_0 at 29.1 GiB (tight). FP8 (31.2 GiB) and BF16 (54.2 GiB) need 48GB-class and 80GB-class hardware respectively.
Two Q4_K_M builds are listed — which one do I get?
The quant label does not decide the size; the build does. This page prices unsloth's Q4_K_M (15.7 GiB, 16,817,244,384 bytes, pinned rev 82d411ac); ggml-org's Q4_K_M measures 17.8 GiB (19,095,766,304 bytes, pinned rev 8a7ee08e) — 13.4% larger at the same name, different tensor coverage. The ollama short form serves its own Q4_K_M build; whichever file you pull, read its bytes, not its label.
How does this site compute these VRAM numbers?
Weights are measured file sizes at pinned revisions (sources below); totals add runtime overhead plus the KV cache at 32K context. Here KV@32K = 2.0 GiB, a reported figure scaled from the native-context measurement (the ~0.07 GiB GDN constant state is unpublished and noted as a slight undercount). A build fits a card at 32K when the total clears usable VRAM (nominal minus reserve), and tight means under 2.5 GiB of headroom — the full formula lives in the method box under the quantization table.