LLM VRAM Calculator
Estimate exactly how much GPU VRAM you need to run any LLM locally. Covers quantization levels, context length, KV cache overhead, and runtime memory. Then maps the result to real GPUs from our comparison data.
Figures reviewed September 15, 2026.
Configure your workload
Parameter count. For MoE models, we use the total parameter count (VRAM must hold all weights).
Lower quantization = smaller VRAM footprint but quality tradeoff. Q4_K_M is the most common default.
KV cache grows linearly with context. Doubling context roughly doubles the cache size.
Q8 KV cache halves memory with minor quality impact. Supported in llama.cpp and newer Ollama.
CUDA context + framework overhead. Ollama and llama.cpp are leanest.
Total VRAM needed
16.4 GB
Recommended VRAM tier
16GB card
A 16GB GPU will handle this workload with ~4GB headroom for longer context or batching.
GPUs that fit this workload
Preset readings
Eight tiers at the defaults — Q4_K_M weights, FP16 cache, 1.5GB runtime — with the smallest card in our comparison data that holds the 4K total. Read straight off; the calculator is for the combinations not listed.
| Tier | Q4_K_M | Q8_0 | FP16 | KV @4K | KV @32K | Total @4K | Smallest card |
|---|---|---|---|---|---|---|---|
| 7B Mistral 7B v0.3 | 3.94 | 7.44 | 14 | 0.5 | 4 | 5.94 | RTX 4060 (8GB) |
| 8B Llama 3.1 8B | 4.5 | 8.5 | 16 | 0.5 | 4 | 6.5 | RTX 4060 (8GB) |
| 13B Llama 2 13B | 7.31 | 13.8 | 26 | 3.13 | 25 | 11.9 | RTX 4060 Ti 16GB |
| 14B Qwen 2.5 14B | 7.88 | 14.9 | 28 | 0.75 | 6 | 10.1 | RTX 3060 12GB (used) |
| 27B Gemma 2 27B | 15.2 | 28.7 | 54 | 1.44 | 11.5 | 18.1 | RTX 3090 (used) (24GB) |
| 32B Qwen 2.5 32B | 18 | 34 | 64 | 1 | 8 | 20.5 | RTX 3090 (used) (24GB) |
| 34B Yi 1.5 34B | 19.1 | 36.1 | 68 | 0.94 | 7.5 | 21.6 | RTX 3090 (used) (24GB) |
| 70B Llama 3.3 70B | 39.4 | 74.4 | 140 | 1.25 | 10 | 42.1 | NVIDIA A6000 (48GB) |
How the math works
The total VRAM needed for running an LLM is the sum of three components: model weights, KV cache, and runtime overhead. Each scales differently based on your configuration.
1. Model weights
Weight memory scales with parameter count and quantization bits per parameter. For a 7B model at Q4_K_M (~4.5 bits/param), the weights occupy roughly 4 GB. At FP16 the same model takes ~14 GB. See our how much VRAM for local LLM guide for a full breakdown from 7B to 70B.
weights_GB = (params_in_billions × bits_per_param) / 8
2. KV cache
Every token of context adds to the KV cache, and its width is set by the model's key/value heads — not by its hidden size, which is the assumption that makes most VRAM estimates wrong. Grouped-query attention shares one set of keys and values across several query heads, so Llama 3 70B holds 8 KV heads of 128 dimensions against 64 query heads: 0.31 MB per token at FP16, about 10 GB at 32K context. Multi-head models from before GQA are far heavier per parameter — Llama 2 13B keeps 40 KV heads and costs 0.78 MB per token, two and a half times a 70B.
kv_GB = 2 × num_layers × (kv_heads × head_dim) × context × precision_bytes / 1024³
3. Runtime overhead
CUDA context, framework buffers, activation memory during inference. Ollama and llama.cpp are leanest (~1-1.5 GB). vLLM with heavy batching can consume 2-3 GB more. Text Generation WebUI with extensions loaded is heaviest.
Why "recommended" differs from "fits"
A workload that needs 15.8 GB will technically fit in a 16GB card, but there's no headroom. Any extra context, a second user, or a driver update can push it over and cause OOM errors. We recommend the next VRAM tier up when headroom is under 2 GB. This is why our Ollama GPU guide and used RTX 3090 guide often point readers toward 24 GB cards once models push past 16 GB.
This calculator gives representative estimates based on typical Transformer architectures. Actual VRAM usage varies by model family (Llama, Mistral, Qwen have slightly different layer counts and hidden sizes), driver, framework, and batch size. Treat results as ±10% accurate. See our methodology for how we derive these numbers.