Qwen3.5-27B VRAM requirements
fits any 24GB card, 4.2 GiB to spare · unsloth's UD-Q5_K_XL tops out at 21.4 GiB, 1.1 to spare (tight)
Qwen3.5-27B needs 16.4 GiB at standard Q4_K_M (unsloth build) with 4K context, the community runtime default, and 18.2 GiB at 32K (KV counted), so any 24GB card runs it with 4.2 GiB to spare. Q4_K_M is also the community default quant, the default answer for most setups. Unsloth's UD-Q5_K_XL fits at 21.4 GiB with 1.1 GiB to spare, tight: KV cache at 32K is community-reported at 2.15 GiB from engine allocation logs (llama.cpp #20610 + koboldcpp #2005, linked in sources, not re-measured by us), so every total carries one reported component.
ollama run qwen3.5:27b — comfortable on any 24GB card, even at 32K (Ollama boots at 4K — /set parameter num_ctx 32768).Qwen3.5-27B is Alibaba's hybrid-linear (GDN) model: 64 layers — 48 linear-attention plus 16 full attention (config.json layer_types). The vendor publishes no per-token KV figure; the working number comes from community runtime allocation logs (llama.cpp issue 20610 + koboldcpp issue 2005 — reported, not re-measured by us; see sources), : 2.15 GiB at 32K — 2,048 MiB of full-attention KV across 32,768 cells (F16) plus 149.62 MiB of linear-layer recurrent state per slot; growth is linear in context on the 16 full-attention layers.
Data verified · sources pinned in Where every number comes from
How much VRAM at each quantization? (weights + overhead + 4K KV, batch 1)
| Quantization | VRAM needed @4K | KV @32K reported i · beyond the @4K column | Smallest tracked fit | Note |
|---|---|---|---|---|
| UD_IQ2_XXS | 8.9 GiB | 2.15 GiB | RTX 3060 12GB · ≤~28K ctx | measured · at 32K: 10.7 GiB |
| IQ2_S | 10.7 GiB | 2.15 GiB | RTX 4060 Ti 16GB | measured · at 32K: 12.4 GiB |
| Q3_K_S | 12.3 GiB | 2.15 GiB | RTX 4060 Ti 16GB | measured · at 32K: 14.1 GiB |
| IQ3_M | 14.0 GiB | 2.15 GiB | RTX 4060 Ti 16GB · ≤~12K ctx | measured · at 32K: 15.8 GiB |
| IQ4_XS | 14.8 GiB | 2.15 GiB | RTX 3090 24GB | measured · at 32K: 16.6 GiB |
| Q4_K_M | 16.5 GiB | 2.15 GiB | RTX 3090 24GB | measured · at 32K: 18.2 GiB |
| FP8 | 29.6 GiB | 2.15 GiB | RTX 5090 32GB · ≤~10K ctx | measured · at 32K: 31.4 GiB |
| Q6_K | 21.8 GiB | 2.15 GiB | RTX 3090 24GB · ≤~15K ctx | measured · at 32K: 23.6 GiB |
| UD_Q5_K_XL | 19.7 GiB | 2.15 GiB | RTX 3090 24GB | measured · at 32K: 21.4 GiB |
GB vs GiB: card names are nominal GB; totals are GiB (230) — a "24GB" card holds 24 GiB, and a file quoted 16,740,812,704 B decimal is the same 15.6 GiB. Caliber →
The 48GB-class doors: a dual-GPU combo (2× 24GB; not one card doubled), a workstation card (RTX 6000 Ada), or a Mac Studio (unified memory; runs, slower tok/s).
How this page computes the numbers (reproduce any cell)
- usable VRAM = nominal minus reserve, per card: 12GB → 10.5 GiB, 16GB → 14.5 GiB, 24GB → 22.5 GiB, 32GB → 30.0 GiB.
- total@32K = weights + overhead + KV@32K; KV@32K = 2.15 GiB (config-derived — 70,451 B/token effective at 32K; hybrid models carry a constant state on top, composition in sources).
- total@4K (main column) = total@32K − (KV@32K − 0.146) × 7/8 — the constant state (0.146 GiB on this hybrid) is not scaled down at shorter context; only the linear part divides by eight.
- The "≤~XK ctx" note = 8K + 32K × (usable − total@8K) ÷ KV@32K — the context length that exactly fills the card (KV grows linearly with context on this model).
- A build fits a card at 32K when total@32K ≤ usable; tight = under 2.5 GiB of headroom, one site-wide threshold.
Is your GPU enough? (find your card: one row tells you)
| Your GPU | VRAM | Biggest quant that fits (ceiling) | How it feels |
|---|---|---|---|
| RTX 3060 12GB | 12 GB | IQ2_XXS (2-bit) · 11.6 GiB | 0.4 GiB spare before KV · ≤~15K ctx with KV counted |
| RTX 4060 Ti 16GB | 16 GB | IQ3_M (3-bit) · 15.8 GiB | 0.2 GiB spare before KV · ≤~12K ctx with KV counted |
| RTX 3090 24GB | 24 GB | Q6_K (6-bit) · 23.6 GiB | 0.4 GiB spare before KV · ≤~15K ctx with KV counted |
| RTX 4090 24GB | 24 GB | Q6_K (6-bit) · 23.6 GiB | 0.4 GiB spare before KV · ≤~15K ctx with KV counted |
| RTX 5090 32GB | 32 GB | FP8 (8-bit) · 31.4 GiB | 0.1 GiB spare before KV · ≤~10K ctx with KV counted |
Quants stack by size: each GPU has exactly one ceiling. Main-column totals are weights + overhead + KV @4K, and the @32K KV is a reported 2.15 GiB (engine allocation logs, linked in sources). Q6_K's base clears the 22.5 GiB usable on a 24GB card by 1.1 GiB, but the KV term exceeds that headroom: with KV counted it fits ≤ ~15K ctx. That is the flip-check zone: the quant table's fit column answers "under what conditions does it fit", each build's smallest card per row, with the context it still fits once KV is counted.
What GPU should you buy for it? (the 24GB sweet spot logic)
| The logic | 24GB is the cheapest tier that runs Q4_K_M with real headroom (18.2 GiB at 32K with KV counted, 4.2 GiB to spare). Below it, 12GB tops out at IQ2_XXS and 16GB at IQ3_M — both only at reduced context (~15K / ~12K, the ladder's feel column prices it). Above it, an RTX 5090 32GB reaches FP8; nice, not necessary. |
| Why it fits | 48 of the 64 layers are GDN linear attention holding a constant state; at 32K the KV term is a reported 2.15 GiB (engine allocation logs), so a 27B-class hybrid fits 24GB at Q4 with room to spare. |
⚠ Tight is a site-wide threshold: under 2.5 GiB of headroom against usable VRAM. Q4_K_M @32K leaves 4.2 GiB on a 24GB card, comfortable. The tight fits: UD-Q5_K_XL totals 21.4 GiB with KV counted, 1.1 GiB to spare. Just past the line: Q5_K_L totals 22.9 GiB with KV counted, 0.4 GiB over the 22.5 GiB usable, a flip-check row; and the Q6_K ceiling row: its 1.1 GiB of base headroom is smaller than the reported 2.15 GiB KV @32K, so at full 32K it doesn't fit — it holds ≤ ~15K ctx with context shortened.
Tracked builds (actual file sizes, not theory)
| Build | Publisher | File size | 24GB @32K | Source |
|---|---|---|---|---|
| UD_IQ2_XXS | unsloth | 8.0 GiB | OK · 10.7 | HF @3221f178 verified |
| IQ2_S | bartowski | 9.8 GiB | OK · 12.4 | HF @d7b113c4 verified |
| Q3_K_S | unsloth | 11.4 GiB | OK · 14.1 | HF @3221f178 verified |
| IQ3_M | bartowski | 13.1 GiB | OK · 15.8 | HF @d7b113c4 verified |
| IQ4_XS | unsloth | 13.9 GiB | OK · 16.6 | HF @3221f178 verified |
| Q4_K_M | unsloth | 15.6 GiB | OK · 18.2 | HF @3221f178 verified |
| Q4_K_M (Bartowski) | bartowski | 16.7 GiB | OK · 19.4 | HF @d7b113c4 verified |
| UD_Q5_K_XL | unsloth | 18.8 GiB | TIGHT · 21.4 | HF @3221f178 verified |
| Q6_K | unsloth | 20.9 GiB | NO · 23.6 | HF @3221f178 verified |
| FP8 | Qwen | 28.7 GiB | NO · 31.4 | HF @97f5941b verified |
The quant label does not decide the size; the build does: bartowski's Q4_K_M measures 16.75 GiB (17,984,872,928 bytes, pinned rev d7b113c4), 7.4% over unsloth's 15.6 GiB at weights precision, and the two builds sit side by side in the table above. Unsloth's ladder is aimed at the same fixed GPU tiers this site anchors on: UD-Q5_K_XL totals 21.4 against 22.5 GiB usable with KV counted, a designed fit, not luck (unsloth model card, linked, reported). Where 24GB ends: UD-Q5_K_XL fits, Q6_K is the flip-check row, and Q8_0 needs 32GB+.
Self-host or API?
Why in / out? Output tokens cost more than input: a prompt is read in one parallel pass, while output is generated token-by-token — costlier to serve. For example, qwen3.5-27b listed at $0.195 in / $1.56 out per 1M tokens in the 2026-09-18 OpenRouter snapshot; blended at a typical chat mix (4 input : 1 output) ≈ $0.47/1M.
API prices float; the crossover moves less than one decade either way — so this page answers the question that doesn't need live prices: how much monthly usage makes self-hosting worth considering at all.
| Monthly usage | Verdict | Why |
|---|---|---|
| < 40M tokens | API | Even the cheapest self-host ($30/mo) never breaks even against the API bill at this usage — don't bother calculating. |
| 40M-150M tokens | Conditional | The decisive variable is whether you already own a 24GB card — amortization dominates the self-host bill. |
| > 150M tokens | Self-host | Per-token API billing stops scaling here; consider a managed dedicated instance too — at 500M+ tokens/mo it often beats DIY. |
Robustness: at 0.45/1M blended the crossover sits near 130 tokens/mo; at 0.70/1M near 40/mo. The three bands hold under ±2.5× API-price and ±2× self-host-cost floats. Self-host cost basis: $800-1900 GPU amortized over 30 months ($27-63/mo) + ~350W × 4h/day × $0.15/kWh (≈ $6/mo). Live prices via OpenRouter (linked, not re-verified by us); the band is a 27B-class reference from the 2026-09-18 OpenRouter listing, not a pinned price for this model.
Where every number comes from (every figure carries its tag)
Totals: measured weights + a linked overhead anchor + community-reported KV (engine allocation logs, linked) — every total carries one reported component, labeled provisional. This page exists for one config: the ~27B dense agent tier × Q4_K–Q5_K quant × 32K context on a 24GB card — the sweet spot where demand (the smallest models that still run agents) meets supply (quant studios designing marginal fits around the same fixed GPU tiers).
| Tag | Slot | What it is | What the verdict can claim |
|---|---|---|---|
| reported | model | Hugging Face config.json (hypothesis-parameter carrier) | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | static | Hugging Face model page | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | kv | KV cache @32K, full-attention component — llama.cpp engine allocation log: 2,048 MiB across 32,768 cells, F16 K/V (bug-report context: what the engine allocated, not a 'runs great' claim) | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | kv | KV cache @32K, linear-layer recurrent state — koboldcpp engine allocation log: 149.62 MiB per slot (same bug-report context; the 65,536 B/token growth constant is derived from these logged totals) | Tight, conditional — stands on the linked source, re-checkable by you. |
| measured | overhead | Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical) | Determinate — "it runs" means it runs. |
| Hugging Face config.json (hypothesis-parameter carrier) | @fc05daec | accessed 2026-09-06 |
| Hugging Face model page | @fc05daec | accessed 2026-09-06 |
| KV cache @32K, full-attention component — llama.cpp engine allocation log: 2,048 MiB across 32,768 cells, F16 K/V (bug-report context: what the engine allocated, not a 'runs great' claim) | latest | accessed 2026-09-12 |
| KV cache @32K, linear-layer recurrent state — koboldcpp engine allocation log: 149.62 MiB per slot (same bug-report context; the 65,536 B/token growth constant is derived from these logged totals) | latest | accessed 2026-09-12 |
| Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical) | latest | accessed 2026-09-05 |
FAQ
Does Qwen3.5-27B fit in 24GB VRAM?
Yes. The standard Q4_K_M build totals 18.2 GiB at 32K context against 22.5 GiB usable on an RTX 3090 24GB or 4090 24GB, 4.2 GiB to spare. The densest build that fits is unsloth's UD_Q5_K_XL at 21.4 GiB, 1.1 GiB to spare, tight.
How much VRAM does Qwen3.5-27B need at Q4_K_M?
16.4 GiB at 4K context and 18.2 GiB at 32K: 15.6 GiB weights + 0.5 GiB runtime overhead + KV (≈0.3 GiB at 4K, 2.15 GiB at 32K — community-reported from engine allocation logs, linked in sources). That is the unsloth build this page measures; the short form (ollama run qwen3.5:27b) pulls the official build, which boots at 4K.
Can a 24GB card run Qwen3.5-27B at Q6_K?
At the page's promised 32K, no with KV counted: Q6_K totals 23.6 GiB at 32K, 1.1 GiB over the 22.5 GiB usable on a 24GB card. At 4K context it fits, tight: 21.7 GiB, 0.8 GiB to spare. The KV figure is community-reported (2.15 GiB at 32K, engine allocation logs, not our bench): counting it, Q6_K holds ≤ ~15K of context on a 24GB card. Plan for ≤ ~15K ctx at 24GB, or step up to a 32GB-class card for the full 32K.
What is the smallest GPU that runs Qwen3.5-27B?
An RTX 3060 12GB tops out at IQ2_XXS against 10.5 GiB usable (≤~15K of context; the smaller UD-IQ2_XXS build trades a size step for reach and holds ~28K on the same card — see the quant table), and a 16GB card reaches IQ3_M against 14.5 GiB usable (≤~12K) — the full 32K window needs 24GB. Q4_K_M is comfortable on any 24GB card.
Does context length change the VRAM number?
The weights + overhead core is context-flat (48 of 64 layers are GDN linear, a constant state), and context growth rides only the 16 full-attention layers. The figure on this page is community-reported: 2.15 GiB at 32K from engine allocation logs — llama.cpp issue 20610 logs 2,048 MiB of full-attention KV across 32,768 cells (F16), and koboldcpp issue 2005 logs 149.62 MiB of recurrent state per slot; bug-report context, so the figures are what the engine allocated, not a 'runs great' claim. The 65,536 B/token growth constant is derived from those logged totals, so shorter context scales the growing part linearly on the 16 full-attention layers while the 48 linear layers' state stays constant. For other contexts or KV schemes, the quant table's fit column gives each build's smallest card, and the GPU Finder answers the cross-model question.
I see Q5_K_XL also fits my 24GB card — why does this page still recommend Q4_K_M?
Because Q4_K_M is the community default: it matches what most people actually download (ollama's default tag, bartowski's 'default for most use cases'), keeps 4.2 GiB of headroom for longer context and everyday workloads, and the quality step from Q4 to Q5 is small next to what extra context buys an agent. UD-Q5_K_XL is a legitimate choice if you want the densest 24GB fit — listed here as a display fact (21.4 GiB, 1.1 GiB to spare, tight), not the recommendation. Quality-critical and short-context? Take the Q5. Most setups: Q4 plus headroom is the steadier default.
How does this site compute these VRAM numbers?
Weights are measured file sizes at pinned revisions (sources below); totals add runtime overhead plus the KV cache at 32K context. Here KV@32K = 2.15 GiB from reported engine allocation logs: 2,048 MiB of full-attention KV plus the 149.62 MiB GDN recurrent state. A build fits a card at 32K when the total clears usable VRAM (nominal minus reserve), and tight means under 2.5 GiB of headroom — the full formula lives in the method box under the quantization table.