Qwen3-4B VRAM requirements
any 12GB card runs it, 3.2 GiB to spare · at 4K context it drops to 3.4 GiB — a 6GB card config
Qwen3-4B needs 3.9 GiB at Q4_K_M (ggml-org build) at 8K context and 7.3 GiB at 32K (classic KV bound counted) — any 12GB card runs it with 3.2 GiB to spare, and at 4K context the whole config is 3.4 GiB, fitting a 6GB card tight. Q4_K_M is the community default. This is the cheapest rung of the 24GB ladder: even the official FP8 build (9.8 GiB at 32K) tops out a 12GB card tight, and F16 (12.5 GiB) fits any 24GB card with 10 GiB of headroom. The same 4B class is also the phone-runnable tier — at 4K it loads in PocketPal-class apps on 8GB phones; the on-phone axis gets its own page family.
ollama run qwen3:4b — comfortable on any 12GB card, even at 32K (Ollama boots at 4K context — /set parameter num_ctx 32768 for the 32K window this page prices).Qwen3-4B is Alibaba's entry dense model from the April 2025 Qwen3 release: 4.0B parameters (GGUF BF16 file-derived), 36 full-attention GQA layers — 32 query heads, 8 KV heads, head dim 128 (config.json) — native 40K context, Apache-2.0. Every layer grows KV linearly, so its 32K KV bound is a plain 4.5 GiB config derivation and this page's totals are labeled upper-bound estimates (actual ≤ value). At this size the buying question splits: a used 8GB-class card covers the 4K config, 12GB covers 32K — and the same weights are the phone-runnable class (PocketPal / Google AI Edge Gallery-class apps) at 4K context.
Data verified · sources pinned in Where every number comes from
How much VRAM at each quantization? (weights + overhead + 4K KV, batch 1)
| Quantization | VRAM needed @4K | KV @32K provisional i · beyond the @4K column | Smallest tracked fit | Note |
|---|---|---|---|---|
| UD_IQ1_S | 2.1 GiB | 4.5 GiB | RTX 3060 12GB | measured · at 32K: 6.0 GiB |
| Q4_K_M | 3.4 GiB | 4.5 GiB | RTX 3060 12GB | measured · at 32K: 7.3 GiB |
| FP8 | 5.9 GiB | 4.5 GiB | RTX 3060 12GB | measured · at 32K: 9.8 GiB |
| F16 | 8.6 GiB | 4.5 GiB | RTX 3060 12GB · ≤~18K ctx | measured · at 32K: 12.5 GiB |
| Q8_0 | 5.0 GiB | 4.5 GiB | RTX 3060 12GB | measured · at 32K: 9.0 GiB |
GB vs GiB: card names are the manufacturer's nominal GB; totals here are GiB (230 bytes) — a "24GB" card holds 24 GiB, and a file quoted 2,501,512,224 B in decimal is the same 2.3 GiB. Caliber →
The 48GB-class doors, spelled out: a dual-GPU combo (2× 24GB; not one card doubled), a workstation card (RTX 6000 Ada), or a Mac Studio (unified memory; runs, slower tok/s). For this model they are all absurd — the entire ladder fits one 24GB card.
How this page computes the numbers (reproduce any cell)
- usable VRAM = nominal minus reserve, per card: 12GB → 10.5 GiB, 16GB → 14.5 GiB, 24GB → 22.5 GiB, 32GB → 30.0 GiB.
- total@32K = weights + overhead + KV@32K; KV@32K = 4.5 GiB (config-derived — 147,456 B/token effective at 32K; hybrid models carry a constant state on top, composition in sources).
- total@4K (main column) = total@32K − KV@32K × 7/8 — KV is fully linear in context on this model.
- The "≤~XK ctx" note = 8K + 32K × (usable − total@8K) ÷ KV@32K — the context length that exactly fills the card (KV grows linearly with context on this model).
- A build fits a card at 32K when total@32K ≤ usable; tight = under 2.5 GiB of headroom, one site-wide threshold.
Is your GPU enough? (find your card: one row tells you)
| Your GPU | VRAM | Biggest quant that fits (ceiling) | How it feels |
|---|---|---|---|
| RTX 3060 12GB | 12 GB | FP8 (8-bit) · 9.8 GiB | 0.7 GiB spare at 32K (incl KV bound) |
| RTX 4060 Ti 16GB | 16 GB | F16 (16-bit) · 12.5 GiB | 2.0 GiB spare at 32K (incl KV bound) |
| RTX 3090 24GB | 24 GB | F16 (16-bit) · 12.5 GiB | 10.0 GiB spare at 32K (incl KV bound) |
| RTX 4090 24GB | 24 GB | F16 (16-bit) · 12.5 GiB | 10.0 GiB spare at 32K (incl KV bound) |
| RTX 5090 32GB | 32 GB | F16 (16-bit) · 12.5 GiB | 17.5 GiB spare at 32K (incl KV bound) |
Quants stack by size: each GPU has exactly one ceiling — a card that runs a bigger build runs every smaller one. Main-column totals price the 4K load (weights + overhead + the KV term at 4K — the community runtime default); each row's Note prices the full 32K load this page promises (Q4_K_M at 32K = 7.3 GiB), and CPU offload trades tok/s for headroom.
What GPU should you buy for it? (the 24GB sweet spot logic)
| The logic | 12GB is the cheapest tier that runs the promised 32K config: Q4_K_M totals 7.3 GiB at the bound, 3.2 GiB to spare on a used RTX 3060 12GB. Below it, 8GB cards run the same build at 4K context (3.4 GiB) — the phone-class footprint — and a 6GB card fits it tight. 24GB buys nothing for this model alone: the entire tracked ladder, F16 included, leaves 10 GiB of headroom there. |
| Why it fits | Dense 4B at Q4_K_M is 2.3 GiB of weights; even with the 4.5 GiB KV bound at 32K the total is 7.3 GiB. Buying up is buying for other models. |
⚠ Tight is a site-wide threshold: under 2.5 GiB of headroom. Q4_K_M @32K leaves 15.2 GiB on a 24GB card — beyond comfortable. The tight rows worth knowing: the 12GB ceiling (official FP8, 9.8 GiB, 0.7 spare) and the 16GB ceiling (F16, 12.5 GiB, 2.0 spare) — both are ceiling rows, not the page's pick. All totals carry the classic KV bound, so these verdicts are stable, not flip-check rows.
Tracked builds (actual file sizes, not theory)
| Build | Publisher | File size | 24GB @32K | Source |
|---|---|---|---|---|
| UD_IQ1_S | unsloth | 1.0 GiB | OK ON 12G · 6.0 | HF @22c9fc8a verified |
| Q4_K_M | ggml-org | 2.3 GiB | OK ON 12G · 7.3 | HF @2f3b082b verified |
| FP8 | Qwen | 4.8 GiB | OK · 9.8 | HF @96b30dc1 verified |
| F16 | ggml-org | 7.5 GiB | OK · 12.5 | HF @2f3b082b verified |
| Q8_0 | ggml-org | 4.0 GiB | OK · 9.0 | HF @2f3b082b verified |
Same quant, different builds — the mirror case again: Q4_K_M and Q8_0 ship at the same size across unsloth, ggml-org, and bartowski (one row stands for all three; the ggml-org file is cited), and no >5% same-name spread exists in this ladder. The one trap this model shares with its phone pages: the ollama blob bundles more than the GGUF files, so read file listings, not tag pages. Where 12GB ends at 32K: the official FP8 build at 9.8 GiB; BF16 (12.5 GiB) is the 16GB door and fits any 24GB card with 10 GiB to spare.
Self-host or API?
API prices are sourced from OpenRouter (linked, not re-verified). We have not pinned a price for Qwen3-4B, so the breakeven cell stays empty rather than guessing. When a price is pinned, this block compares API cost per million tokens against self-hosting on your own card.
Where every number comes from (every figure carries its tag)
Totals: measured weights + KV as the classic stacked upper bound (plain-GQA config derivation — labeled, and actual ≤ value) + measured overhead. This page exists for one config: the 4B dense tier × Q4_K-class quant × 32K context — the cheapest rung of the ladder, where the GPU question turns into "which old small card do you already own".
| Tag | Slot | What it is | What the verdict can claim |
|---|---|---|---|
| reported | model | Hugging Face config.json (hypothesis-parameter carrier) | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | static | Hugging Face model page | Tight, conditional — stands on the linked source, re-checkable by you. |
| provisional | kv | KV cache @ 32K — no published measurement for Qwen3-4B in our sources (kaitchup/vLLM/official reports checked 2026-09-14); the page shows the classic stacked bound instead: 147,456 B/token × 32K = 4.5 GiB, config-derived (36 layers × 8 KV heads × 128 dim × K/V × 2 bytes), actual ≤ value | Very likely right, order of magnitude only — replaced the moment measured values land. |
| measured | overhead | Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | Determinate — "it runs" means it runs. |
| Hugging Face config.json (hypothesis-parameter carrier) | @1cfa9a72 | accessed 2026-09-14 |
| Hugging Face model page | @1cfa9a72 | accessed 2026-09-14 |
| KV cache @ 32K — no published measurement for Qwen3-4B in our sources (kaitchup/vLLM/official reports checked 2026-09-14); the page shows the classic stacked bound instead: 147,456 B/token × 32K = 4.5 GiB, config-derived (36 layers × 8 KV heads × 128 dim × K/V × 2 bytes), actual ≤ value | latest | — |
| Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | latest | accessed 2026-09-05 |
FAQ
Does Qwen3-4B fit in 8GB VRAM?
Yes at 4K context, and in 12GB at the page's full 32K. The Q4_K_M build totals 7.3 GiB at 32K context (weights + overhead + the 4.5 GiB classic KV bound) — any 12GB card runs it with 3.2 GiB to spare. At 4K context the KV term drops eightfold and the same build is 3.4 GiB, fitting an 8GB card with room and a 6GB card tight.
How much VRAM does Qwen3-4B need at Q4_K_M?
7.3 GiB at 32K context: 2.3 GiB weights + 0.5 GiB runtime overhead + 4.5 GiB KV cache. The KV figure is the classic stacked bound for this plain-GQA model (147,456 bytes/token × 32,768 tokens — 36 layers × 8 KV heads × 128 dim, config-derived, unmeasured): actual KV ≤ that value, and KV quantization shrinks it further. At 4K context the KV term drops eightfold to ≈0.6 GiB, so the same build runs in ≈3.4 GiB. The short form (ollama run qwen3:4b) pulls the ollama Q4_K_M build.
Can Qwen3-4B run on a phone?
Yes — this 4B dense class is the phone-runnable tier. At 4K context the full Q4_K_M config is 3.4 GiB of RAM: inside the ~60% usable rule, an 8GB phone carries it (tight), a 12GB phone comfortably. Use PocketPal or llama.cpp for the GGUF build, or Google AI Edge Gallery-class runtimes. This page prices the GPU question; the on-phone axis is the companion page family in the same database. Verify the app loads this exact file before a long download.
What is the smallest GPU that runs Qwen3-4B at 32K?
An RTX 3060 12GB tops out at the official FP8 build (9.8 GiB at the bound, 0.7 spare, tight) — Q4_K_M itself is comfortable there. A 16GB card reaches F16 (12.5 GiB, 2.0 spare, tight). There is no smaller drama anywhere: even F16 fits any 24GB card with 10 GiB of headroom.
Which Q4_K_M file should I download?
For this model it doesn't matter by size: Q4_K_M ships at the same size across unsloth, ggml-org, and bartowski (one row stands for all three in the table, the ggml-org file cited; we compare sizes, not hashes), and the same is true of Q8_0 — no >5% same-name spread exists here. If a guide hands you a specific file, any of the three works; read bytes, not labels. The one caveat is the ollama blob, which bundles more than the GGUF files — read file listings, not tag pages.
Should I buy a GPU for Qwen3-4B at all?
Probably not a new one. The promised 32K config runs on a used RTX 3060 12GB (3.2 GiB spare), and the 4K config fits hardware most people already own — an 8GB card or an 8GB phone. Buy up only if you are also targeting the 14B/27B pages in the related tables; for this model alone, the cheapest working tier is the answer.
How does this site compute these VRAM numbers?
Weights are measured file sizes at pinned revisions (sources below); totals add runtime overhead plus the KV cache at 32K context. Here KV@32K = 4.5 GiB = 147,456 B/token × 32,768 (36 layers × 8 KV heads × 128 dim, config-derived bound). A build fits a card at 32K when the total clears usable VRAM (nominal minus reserve), and tight means under 2.5 GiB of headroom — the full formula lives in the method box under the quantization table.