Qwen3.8-27B VRAM requirements
fits any 24GB card, 3.7 GiB to spare · official FP8 needs 31.3 GiB
Qwen3.8-27B needs 17.3 GiB at Q4_K_M (bartowski build) at 8K context and 18.8 GiB at 32K (reported KV counted) — any 24GB card (RTX 3090 or 4090) runs it with 3.7 GiB to spare. The official FP8 build needs 31.3 GiB and a 48GB-class card at 32K. 12–16GB cards run unsloth's dynamic low-bit builds — 1-bit comfortable on 12GB, 3-bit the ceiling on 16GB.
ollama run hf.co/bartowski/Qwen3.8-27B-GGUF:Q4_K_M — comfortable on any 24GB card, even at 32K (Ollama boots at 4K context — /set parameter num_ctx 32768 in the session to get the full window).ollama run qwen3.8:27b pulls the official build; this page measures the bartowski file. Buying for it? On a budget, price a used RTX 3090 first.Qwen3.8-27B is Alibaba's hybrid-linear model; per its own model card the 27B in the name counts the language model, with the built-in vision tower bringing the total to 27.8B. The official config.json declares a vision_config, and the GGUF repos ship the projector separately as mmproj files (not counted in the sizes here). Its 48 linear-attention plus 16 full-attention layers (64 total, per the same config.json layer_types) are why its KV cache stays around 2.1 GiB even at 32K context (config-derived: 65,536 bytes/token across the 16 full-attention layers, consistent with the published SGLang calculator — plus a small linear-attention state): the reason a 27B-class model fits 24GB at all. Open weights, Apache-2.0, 256K context.
Data verified · sources pinned in Where every number comes from
How much VRAM at each quantization? (weights + overhead + 4K KV, batch 1)
| Quantization | VRAM needed @4K | KV @32K reported i · beyond the @4K column | Smallest tracked fit | Note |
|---|---|---|---|---|
| IQ2_S | 9.8 GiB | 2.073 GiB | RTX 3060 12GB · ≤~14K ctx | measured · at 32K: 11.6 GiB |
| IQ3_XS | 12.7 GiB | 2.073 GiB | RTX 4060 Ti 16GB | measured · at 32K: 14.5 GiB |
| Q4_K_M | 17.1 GiB | 2.073 GiB | RTX 3090 24GB | measured · at 32K: 18.8 GiB |
| Q5_K_M | 20.3 GiB | 2.073 GiB | RTX 3090 24GB | measured · at 32K: 22.1 GiB |
| Q6_K | 23.0 GiB | 2.073 GiB | RTX 5090 32GB | measured · at 32K: 24.8 GiB |
| Q8_0 | 29.4 GiB | 2.073 GiB | RTX 5090 32GB · ≤~14K ctx | measured · at 32K: 31.1 GiB |
| FP8 | 29.6 GiB | 2.073 GiB | RTX 5090 32GB · ≤~11K ctx | measured · at 32K: 31.3 GiB |
| BF16 | 52.6 GiB | 2.073 GiB | — | measured · at 32K: 54.3 GiB |
GB vs GiB: card names are the manufacturer's nominal GB; totals here are GiB (230 bytes) — a "24GB" card holds 24 GiB, and a file quoted 17.4 GB in decimal is the same 16.2 GiB. Caliber →
The 48GB-class doors, spelled out: a dual-GPU combo (2× 24GB; not one card doubled: topology, power and tensor-parallel split overhead apply), a workstation card (RTX 6000 Ada), or a Mac Studio (unified memory; runs, slower tok/s).
How this page computes the numbers (reproduce any cell)
- usable VRAM = nominal minus reserve, per card: 12GB → 10.5 GiB, 16GB → 14.5 GiB, 24GB → 22.5 GiB, 32GB → 30.0 GiB.
- total@32K = weights + overhead + KV@32K; KV@32K = 2.073 GiB (config-derived — 67,928 B/token effective at 32K; hybrid models carry a constant state on top, composition in sources).
- total@4K (main column) = total@32K − (KV@32K − 0.073) × 7/8 — the constant state (0.073 GiB on this hybrid) is not scaled down at shorter context; only the linear part divides by eight.
- The "≤~XK ctx" note = 8K + 32K × (usable − total@8K) ÷ KV@32K — the context length that exactly fills the card (KV grows linearly with context on this model).
- A build fits a card at 32K when total@32K ≤ usable; tight = under 2.5 GiB of headroom, one site-wide threshold.
Is your GPU enough? (find your card: one row tells you)
| Your GPU | VRAM | Biggest quant that fits (ceiling) | How it feels |
|---|---|---|---|
| RTX 3060 12GB | 12 GB | UD_IQ2_S (2-bit) · 10.4 GiB | 0.1 GiB spare at 32K (KV counted) |
| RTX 4060 Ti 16GB | 16 GB | IQ3_XS (3-bit) · 14.5 GiB | 0.01 GiB spare at 32K (KV counted) |
| RTX 3090 24GB | 24 GB | Q5_K_M (5-bit) · 22.1 GiB | 0.4 GiB spare at 32K (KV counted) |
| RTX 4090 24GB | 24 GB | Q5_K_M (5-bit) · 22.1 GiB | 0.4 GiB spare at 32K (KV counted) |
| RTX 5090 32GB | 32 GB | Q6_K (6-bit) · 24.8 GiB | 5.2 GiB spare at 32K (KV counted) |
Quants stack by size within the GGUF ladder: a card that runs a bigger build runs every smaller one. So each GPU has exactly one ceiling — the largest build that fits, whichever publisher's — and this table names it: nothing else to cross-reference. Ceilings are full-GPU at 32K — every layer on the card; shorter context raises them a step, and CPU offload (llama.cpp --n-cpu-moe) runs bigger quants partially in system RAM, trading tok/s.
What GPU should you buy for it? (the 24GB sweet spot logic)
| The logic | 24GB is the cheapest tier that runs Q4_K_M comfortably (18.8 GiB at 32K, 3.7 GiB to spare). Below it, 12–16GB cards save money but top out at 2-bit to 3-bit quants. Above it, a 32GB RTX 5090 buys Q6_K comfortably (Q8_0 totals 31.1 GiB — 48GB-class territory); nice, not necessary. |
| Why it fits | Hybrid-linear attention keeps KV cache at 2.1 GiB even at 32K context; a full-attention 28B would not come close to 24GB |
| Source | bartowski GGUF (pinned) + official config · 2026-09 · official files verified |
⚠ Tight is a site-wide threshold: under 2.5 GiB of headroom. This config leaves 3.7 GiB at 32K (comfortable). The tightest fit on 24GB is the Q5_K_M build: 22.1 GiB, about 0.4 GiB to spare.
Tracked builds (actual file sizes, not theory)
| Build | Publisher | File size | 24GB @32K | Source |
|---|---|---|---|---|
| IQ2_S | bartowski | 9.0 GiB | OK · 11.6 | HF @0c92138c verified |
| IQ3_XS | bartowski | 11.9 GiB | OK · 14.5 | HF @0c92138c verified |
| Q4_K_M | bartowski | 16.2 GiB | OK · 18.8 | HF @0c92138c verified |
| Q5_K_M | bartowski | 19.5 GiB | TIGHT · 22.1 | HF @0c92138c verified |
| Q6_K | bartowski | 22.2 GiB | NO · 24.8 | HF @0c92138c verified |
| Q8_0 | ggml-org | 28.5 GiB | NO · 31.1 | HF @efbb3b1f verified |
| FP8 | Qwen | 28.7 GiB | NO · 31.3 | HF @017b9c7a verified |
| BF16 | Qwen | 51.7 GiB | NO · 54.3 | HF @1d4bf0f2 verified |
Q4 is the sweet spot this page anchors on, and the build decides the size within it. At Q4_K_M the spread between publishers is real: unsloth's dynamic build (15.3 GiB) vs bartowski's canonical (16.2 GiB) vs ggml-org's (17.7 GiB) — 2.3 GiB on the same quant name. unsloth's UD-Q4 family is calibrated against the standard Q4_K_* ladder: UD Q4_K_S (14.3 GiB) undercuts every standard Q4 build, and UD Q4_K_XL (16.4 GiB) reaches Q4_K_L territory for less than bartowski's Q4_K_L (17.5 GiB). At the smaller Q4 names the publishers mirror each other: Q4_0 and Q4_1 differ by 2%/2% between unsloth and bartowski (same-size mirror builds; the table shows the tracked one). At 32K on a 24GB card, everything through Q5_K_M fits; the two standard Q4_K_M builds fit with room (18.8 and 20.3 GiB).
Self-host or API?
Why in / out? Output tokens cost more than input: a prompt is read in one parallel pass, while output is generated token-by-token — costlier to serve. For example, qwen3.8-27b listed at $0.214 in / $2.55 out per 1M tokens in the 2026-09-18 OpenRouter snapshot; blended at a typical chat mix (4 input : 1 output) ≈ $0.68/1M.
API prices float; the crossover moves less than one decade either way — so this page answers the question that doesn't need live prices: how much monthly usage makes self-hosting worth considering at all.
| Monthly usage | Verdict | Why |
|---|---|---|
| < 40M tokens | API | Even the cheapest self-host ($30/mo) never breaks even against the API bill at this usage — don't bother calculating. |
| 40M-150M tokens | Conditional | The decisive variable is whether you already own a 24GB card — amortization dominates the self-host bill. |
| > 150M tokens | Self-host | Per-token API billing stops scaling here; consider a managed dedicated instance too — at 500M+ tokens/mo it often beats DIY. |
Robustness: at 0.45/1M blended the crossover sits near 130 tokens/mo; at 0.70/1M near 40/mo. The three bands hold under ±2.5× API-price and ±2× self-host-cost floats. Self-host cost basis: $800-1900 GPU amortized over 30 months ($27-63/mo) + ~350W × 4h/day × $0.15/kWh (≈ $6/mo). Live prices via OpenRouter (linked, not re-verified by us); the band is a 27B-class reference from the 2026-09-18 OpenRouter listing, not a pinned price for this model.
Where every number comes from (every figure carries its tag)
Totals: measured weights + reported KV (per-token constant config-derived from the official config.json) + measured overhead — labeled, additive.
| Tag | Slot | What it is | What the verdict can claim |
|---|---|---|---|
| reported | model | Hugging Face config.json (layer_types, vision_config, kv heads/dims — hypothesis-parameter carrier) | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | static | Hugging Face model page | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | kv | KV cache @32K — 65,536 bytes/token config-derived from the official config.json (2 × 4 KV heads × 256 head_dim × 2 bytes × 16 full-attention layers), consistent with the SGLang cookbook: 2.0 GiB across the 16 full-attention layers + GDN linear-attention constant state 78.4 MB published by SGLang (= 2.073 GiB); growth is linear in context on the full-attention layers | Tight, conditional — stands on the linked source, re-checkable by you. |
| measured | overhead | Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | Determinate — "it runs" means it runs. |
| Hugging Face config.json (layer_types, vision_config, kv heads/dims — hypothesis-parameter carrier) | @1d4bf0f2 | accessed 2026-08-31 |
| Hugging Face model page | @1d4bf0f2 | accessed 2026-09-15 |
| KV cache @32K — 65,536 bytes/token config-derived from the official config.json (2 × 4 KV heads × 256 head_dim × 2 bytes × 16 full-attention layers), consistent with the SGLang cookbook: 2.0 GiB across the 16 full-attention layers + GDN linear-attention constant state 78.4 MB published by SGLang (= 2.073 GiB); growth is linear in context on the full-attention layers | latest | accessed 2026-09-05 |
| Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | latest | accessed 2026-09-05 |
FAQ
Does Qwen3.8-27B fit in 24GB VRAM?
Yes. The bartowski Q4_K_M build totals 18.8 GiB at 32K context against 22.5 GiB usable on an RTX 3090 24GB or 4090 24GB — 3.7 GiB to spare. The unsloth dynamic Q5_K_M (21.0 GiB) and the standard bartowski Q5_K_M (22.1 GiB) also fit, tight. Official FP8 (31.3 GiB) does not: it needs 48GB-class hardware at 32K.
How much VRAM does Qwen3.8-27B need at Q4_K_M?
18.8 GiB at 32K context: 16.2 GiB weights + 0.5 GiB runtime overhead + 2.1 GiB KV cache. The KV figure is config-derived from the official config.json (65,536 bytes/token across the 16 full-attention layers, consistent with the SGLang calculator), plus the linear-attention constant state. At 4K context the linear part of the KV term shrinks eightfold (the constant state does not scale), landing at ≈0.3 GiB, so the same build runs in ≈17.1 GiB. The short form (ollama run qwen3.8:27b) pulls the ollama Q4_K_M build.
What is the smallest GPU that runs Qwen3.8-27B?
At 32K context an RTX 3060 12GB tops out at UD_IQ2_S (fits, tight); at 8K context the smallest build (UD_IQ2_XXS) drops to 7.8 GiB and fits with headroom. A 16GB card reaches IQ3_XS. Q4_K_M, the standard 4-bit build, needs a 24GB card.
Can an RTX 4090 run Qwen3.8-27B above Q4?
Not by a standard build at 32K: Q6_K totals 24.8 GiB, 2.3 GiB over the 22.5 GiB usable — at 8K context it still misses, 23.3 GiB against 22.5 GiB usable. The 24GB ceiling at 32K is the Q5_K_M build at 22.1 GiB (0.4 GiB to spare, tight). An RTX 5090 32GB tops out at Q6_K — 24.8 GiB at 32K, 5.2 GiB to spare; Q8_0 totals 31.1 GiB, so it and official FP8 need 48GB-class.
Three Q4_K_M builds exist — which one do I get?
The quant label does not decide the size; the build does. This page prices bartowski's Q4_K_M (16.2 GiB, pinned at the revision in the sources table); unsloth's dynamic Q4_K_M measures 15.3 GiB and ggml-org's 17.7 GiB — 2.3 GiB of spread at the same quant name, different tensor coverage. The ollama short form serves its own Q4_K_M build; whichever file you pull, read its bytes, not its label.
Why is the KV cache only 2.1 GiB and not the classic full-attention estimate?
Because 48 of the 64 layers are linear attention (GDN) and hold a constant recurrent state instead of growing KV. The 2.1 GiB @32K is the bound for the 16 full-attention layers only (65,536 bytes/token config-derived from the official config.json, consistent with the SGLang calculator, linked) × 32K tokens, plus a small constant state for the linear layers. A classic all-layers bound would overstate what the 16 layers actually hold.
How does this site compute these VRAM numbers?
Weights are measured file sizes at pinned revisions (sources below); totals add runtime overhead plus the KV cache at 32K context. Here KV@32K = 2.073 GiB: 65,536 B/token across the 16 full-attention layers, plus the GDN linear layers' 0.073 GiB constant state (SGLang's published constant). A build fits a card at 32K when the total clears usable VRAM (nominal minus reserve), and tight means under 2.5 GiB of headroom — the full formula lives in the method box under the quantization table.