Mistral Small 3.1 VRAM requirements
fits any 24GB card, 3.7 GiB to spare · the 24GB ceiling is UD-Q5_K_XL at 21.1 GiB, tight
Mistral Small 3.1 needs 15.1 GiB at Q4_K_M (unsloth build) at 8K context and 18.8 GiB at 32K (classic KV bound counted) — any 24GB card runs it with 3.7 GiB to spare, which is why the community calls this the 24GB card's best friend. Q4_K_M is the community default. The ceiling on 24GB is unsloth's UD-Q5_K_XL at 21.1 GiB (1.4 to spare, tight), and Q6-class needs more room: UD-Q6_K_XL totals 24.9 GiB at the bound — 2.4 over 22.5 usable, fitting 24GB only with context shortened to ~9K. The nearest miss in the database lives on this page too: the smallest 1-bit build totals 10.7 GiB at 32K, 0.2 GiB over a 12GB card's 10.5 usable — at 8K context it fits (6.9 GiB), tight.
ollama run mistral-small3.1 — comfortable on any 24GB card, even at 32K (Ollama boots at 4K context — /set parameter num_ctx 32768 for the full window).Mistral Small 3.1 is Mistral AI's March 2025 refresh of the Small line: a 24B dense GQA model — 40 full-attention layers, 32 query heads, 8 KV heads, head dim 128 (config.json text_config) — with a vision tower riding along (multimodal wrapper; the GGUF repos ship mmproj separately). Native 128K context, Apache-2.0. Every layer grows KV linearly, so its 32K KV bound is the same 5.0 GiB config derivation the 40-layer GQA arithmetic gives — and this page's totals are labeled upper-bound estimates (actual ≤ value).
Data verified · sources pinned in Where every number comes from
How much VRAM at each quantization? (weights + overhead + 4K KV, batch 1)
| Quantization | VRAM needed @4K | KV @32K provisional i · beyond the @4K column | Smallest tracked fit | Note |
|---|---|---|---|---|
| UD_IQ1_S | 6.3 GiB | 5 GiB | RTX 3060 12GB · ≤~31K ctx | measured · at 32K: 10.7 GiB |
| UD_IQ3_XXS | 9.9 GiB | 5 GiB | RTX 4060 Ti 16GB | measured · at 32K: 14.3 GiB |
| Q4_K_M | 14.5 GiB | 5 GiB | RTX 3090 24GB | measured · at 32K: 18.8 GiB |
| UD_Q5_K_XL | 16.7 GiB | 5 GiB | RTX 3090 24GB | measured · at 32K: 21.1 GiB |
| Q8_0 | 24.5 GiB | 5 GiB | RTX 5090 32GB | measured · at 32K: 28.8 GiB |
GB vs GiB: card names are the manufacturer's nominal GB; totals here are GiB (230 bytes) — a "24GB" card holds 24 GiB, and a file quoted 14,334,000,000 B in decimal is the same 13.4 GiB. Caliber →
The 48GB-class doors, spelled out: a dual-GPU combo (2× 24GB; not one card doubled), a workstation card (RTX 6000 Ada), or a Mac Studio (unified memory; runs, slower tok/s). Here they only matter for BF16 — every quant readers actually download fits one 24GB card.
How this page computes the numbers (reproduce any cell)
- usable VRAM = nominal minus reserve, per card: 12GB → 10.5 GiB, 16GB → 14.5 GiB, 24GB → 22.5 GiB, 32GB → 30.0 GiB.
- total@32K = weights + overhead + KV@32K; KV@32K = 5 GiB (config-derived — 163,840 B/token effective at 32K; hybrid models carry a constant state on top, composition in sources).
- total@4K (main column) = total@32K − KV@32K × 7/8 — KV is fully linear in context on this model.
- The "≤~XK ctx" note = 8K + 32K × (usable − total@8K) ÷ KV@32K — the context length that exactly fills the card (KV grows linearly with context on this model).
- A build fits a card at 32K when total@32K ≤ usable; tight = under 2.5 GiB of headroom, one site-wide threshold.
Is your GPU enough? (find your card: one row tells you)
| Your GPU | VRAM | Biggest quant that fits (ceiling) | How it feels |
|---|---|---|---|
| RTX 3060 12GB | 12 GB | — | — |
| RTX 4060 Ti 16GB | 16 GB | UD_IQ3_XXS (3-bit) · 14.3 GiB | 0.2 GiB spare at 32K (incl KV bound) |
| RTX 3090 24GB | 24 GB | UD_Q5_K_XL (5-bit) · 21.1 GiB | 1.4 GiB spare at 32K (incl KV bound) |
| RTX 4090 24GB | 24 GB | UD_Q5_K_XL (5-bit) · 21.1 GiB | 1.4 GiB spare at 32K (incl KV bound) |
| RTX 5090 32GB | 32 GB | Q8_0 (8-bit) · 28.8 GiB | 1.2 GiB spare at 32K (incl KV bound) |
Quants stack by size: each GPU has exactly one ceiling — a card that runs a bigger build runs every smaller one. Main-column totals price the 4K load (weights + overhead + the KV term at 4K — the community runtime default); each row's Note prices the full 32K load this page promises (the 12GB near-miss flips to a fit at 8K), and CPU offload trades tok/s for headroom.
What GPU should you buy for it? (the 24GB sweet spot logic)
| The logic | 24GB is the tier this model was built around: Q4_K_M totals 18.8 GiB at 32K with the KV bound counted, 3.7 GiB to spare — and the ceiling build (UD-Q5_K_XL, 21.1 GiB) still fits the same cards. Below it, 16GB runs Q4_K_M only at 4K context (14.5 GiB, a knife-edge) and 12GB misses even the 1-bit build at 32K — by 0.2 GiB, the nearest miss on this site. Above it, a 32GB RTX 5090 reaches Q8_0; nice, not necessary — the community's 24GB pick earns the title at 24GB. |
| Why it fits | Dense 24B at Q4_K_M is 13.4 GiB of weights; the 40-layer GQA KV bound adds 5.0 GiB at 32K, totaling 18.8 GiB against 22.5 GiB usable — the 24GB sweet spot with the quality headroom Q4 leaves for longer context and everyday workloads. |
⚠ Tight is a site-wide threshold: under 2.5 GiB of headroom. Q4_K_M @32K leaves 3.7 GiB on 24GB — comfortable by a margin. The tight fits worth knowing: the ceiling build UD-Q5_K_XL totals 21.1 GiB (1.4 to spare), and UD-Q6_K_XL totals 24.9 GiB — 2.4 GiB over the 22.5 GiB usable, fitting 24GB only with context shortened (≈9K at the bound). The 12GB near-miss: UD_IQ1_S totals 10.7 GiB, 0.2 over 10.5 usable — flips to a fit at 8K context. All totals carry the classic KV bound, so these verdicts are stable, not flip-check rows.
Tracked builds (actual file sizes, not theory)
| Build | Publisher | File size | 24GB @32K | Source |
|---|---|---|---|---|
| UD_IQ1_S | unsloth | 5.2 GiB | OK · 10.7 | HF @d63ca941 verified |
| UD_IQ3_XXS | unsloth | 8.8 GiB | OK · 14.3 | HF @d63ca941 verified |
| Q4_K_M | unsloth | 13.3 GiB | OK · 18.8 | HF @d63ca941 verified |
| UD_Q5_K_XL | unsloth | 15.6 GiB | TIGHT · 21.1 | HF @d63ca941 verified |
| Q8_0 | unsloth | 23.3 GiB | NO · 28.8 | HF @d63ca941 verified |
| BF16 | mistralai | 44.7 GiB | NO · 50.2 | HF @68faf511 verified |
unsloth ships the full standard + UD dynamic range for this model — and bartowski ships a ladder too: re-probed on the primary channel 2026-09-15 (27 builds, tracked in our registry). This table stays curated — Q4-first picks, not the whole roster — and same-name spreads follow the >5% rule: same quant name, different build, different bytes. The BF16 row is Mistral's own repo, ten safetensors shards at a pinned revision (the consolidated.safetensors duplicate in the same repo is excluded by the shard-sum rule). Where 24GB ends at 32K: UD-Q5_K_XL at 21.1 GiB; UD-Q6_K_XL (24.9 GiB) is the first door that misses, and Q8_0 (28.8 GiB) needs a 32GB card.
Self-host or API?
Why in / out? Output tokens cost more than input: a prompt is read in one parallel pass, while output is generated token-by-token — costlier to serve. For example, mistral-small-3.1-24b-instruct listed at $0.351 in / $0.555 out per 1M tokens in the 2026-09-18 OpenRouter snapshot; blended at a typical chat mix (4 input : 1 output) ≈ $0.39/1M.
API prices float; the crossover moves less than one decade either way — so this page answers the question that doesn't need live prices: how much monthly usage makes self-hosting worth considering at all.
| Monthly usage | Verdict | Why |
|---|---|---|
| < 50M tokens | API | Even the cheapest self-host ($30/mo) never breaks even against the API bill at this usage — don't bother calculating. |
| 50M-200M tokens | Conditional | The decisive variable is whether you already own a 24GB card — amortization dominates the self-host bill. |
| > 200M tokens | Self-host | Per-token API billing stops scaling here; consider a managed dedicated instance too — at 600M+ tokens/mo it often beats DIY. |
Robustness: at 0.35/1M blended the crossover sits near 170 tokens/mo; at 0.55/1M near 55/mo. The three bands hold under ±2.5× API-price and ±2× self-host-cost floats. Self-host cost basis: $800-1900 GPU amortized over 30 months ($27-63/mo) + ~350W × 4h/day × $0.15/kWh (≈ $6/mo). Live prices via OpenRouter (linked, not re-verified by us); the band is a 24B-class reference from the 2026-09-18 OpenRouter listing, not a pinned price for this model.
Where every number comes from (every figure carries its tag)
Totals: measured weights + KV as the classic stacked upper bound (plain-GQA config derivation — labeled, and actual ≤ value) + measured overhead. This page exists for one config: the 24B dense tier × Q4_K-class quant × 32K context on a 24GB card — the tier the community already voted for, now priced build by build.
| Tag | Slot | What it is | What the verdict can claim |
|---|---|---|---|
| reported | model | HF config.json (hypothesis-parameter carrier) | Tight, conditional — stands on the linked source, re-checkable by you. |
| reported | static | HF model page | Tight, conditional — stands on the linked source, re-checkable by you. |
| provisional | kv | KV cache @ 32K — no published measurement for Mistral Small 3.1 in our sources (checked 2026-09-14); the page shows the classic stacked bound instead: 163,840 B/token × 32K = 5.0 GiB, config-derived (40 layers × 8 KV heads × 128 dim × K/V × 2 bytes), actual ≤ value | Very likely right, order of magnitude only — replaced the moment measured values land. |
| measured | overhead | Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | Determinate — "it runs" means it runs. |
| HF config.json (hypothesis-parameter carrier) | @68faf511 | accessed 2026-09-21 |
| HF model page | @68faf511 | accessed 2026-09-21 |
| KV cache @ 32K — no published measurement for Mistral Small 3.1 in our sources (checked 2026-09-14); the page shows the classic stacked bound instead: 163,840 B/token × 32K = 5.0 GiB, config-derived (40 layers × 8 KV heads × 128 dim × K/V × 2 bytes), actual ≤ value | latest | — |
| Deployment overhead — llama.cpp FA-on compute buffer anchor (507 MiB; measured empirical; same constant as the other champion pages) | latest | accessed 2026-09-05 |
FAQ
Does Mistral Small 3.1 fit in 24GB VRAM?
Yes, comfortably at the community default. The Q4_K_M build totals 18.8 GiB at 32K context (weights + overhead + the 5.0 GiB classic KV bound) against 22.5 GiB usable on an RTX 3090 24GB or 4090 24GB — 3.7 GiB to spare. Even the ceiling build, unsloth's UD-Q5_K_XL at 21.1 GiB, fits the same cards, tight. This is the tier the model’s 24GB reputation is built on.
How much VRAM does Mistral Small 3.1 need at Q4_K_M?
18.8 GiB at 32K context: 13.4 GiB weights + 0.5 GiB runtime overhead + 5.0 GiB KV cache. The KV figure is the classic stacked bound for this plain-GQA model (163,840 bytes/token × 32,768 tokens — 40 layers × 8 KV heads × 128 dim, config-derived, unmeasured): actual KV ≤ that value, and KV quantization shrinks it further. At 4K context the KV term drops eightfold to ≈0.6 GiB, so the same build runs in ≈14.5 GiB — the 16GB-card config, a knife-edge (see the 16GB question below). The short form (ollama run mistral-small3.1) pulls the ollama Q4_K_M build.
Is Mistral Small 3.1 the best 24GB model?
On the numbers this page prices: it is the comfortable fit — Q4_K_M at 18.8 GiB leaves 3.7 GiB of headroom where the 27B hybrids leave 3.9–4.3, with a quality ceiling build (UD-Q5_K_XL) still inside 24GB. "Best" is a workload question beyond VRAM, but the 24GB buying question has a clean answer here: this model, Q4_K_M, one used 3090.
What is the smallest GPU that runs Mistral Small 3.1?
At 32K context nothing fits a 12GB card — the smallest tracked build (UD_IQ1_S) totals 10.7 GiB at the bound, 0.2 GiB over 10.5 usable: the nearest miss on this site (at 8K context that build is 6.9 GiB and fits, tight). A 16GB card reaches UD_IQ3_XXS (14.3 GiB, 0.2 spare). The hero Q4_K_M build needs 24GB at 32K — or 16GB at 4K context, where it totals 14.5 GiB. For Q4_K_M at the promised 32K, buy 24GB.
Can a 16GB card run Mistral Small 3.1 at Q4_K_M?
Yes, with context shortened: at 4K context Q4_K_M totals 14.5 GiB against 14.5 GiB usable on a 16GB card — a knife-edge at the bound: 0.03 GiB of spare on paper, and because the KV figure is an upper bound, anything it saves is real headroom. At the page’s promised 32K it totals 18.8 GiB and does not fit 16GB; the 32K ceiling there is UD_IQ3_XXS. Short-context chat and agent loops fit the 16GB config; long-document work wants 24GB.
Where did the bartowski builds go?
Nowhere — they were unreadable, not absent. During a 2026-09 network outage we read this model’s listings via a mirror channel that carried no bartowski repo; re-probed on the primary channel 2026-09-15, bartowski ships the full ladder (27 builds) and it is tracked in our registry. The table above stays curated — Q4-first picks, with same-name spreads priced under the >5% rule — so a bartowski build joins the display only when it wins a slot, not by default.
How does this site compute these VRAM numbers?
Weights are measured file sizes at pinned revisions (sources below); totals add runtime overhead plus the KV cache at 32K context. Here KV@32K = 5.0 GiB = 163,840 B/token × 32,768 (40 GQA layers, config-derived bound). A build fits a card at 32K when the total clears usable VRAM (nominal minus reserve), and tight means under 2.5 GiB of headroom — the full formula lives in the method box under the quantization table.