| Build of Mistral Large 4 | Size on disk |
|---|---|
| Official FP8 checkpoint (Mistral) | 1.06TB |
| Unsloth Q8_0 GGUF | 1.12TB |
| Unsloth UD-Q4_K_XL GGUF | 605GB |
| Unsloth UD-Q2_K_XL GGUF | 378GB |
| Unsloth UD-IQ2_XXS GGUF | 302GB |
| Unsloth UD-IQ1_S GGUF (smallest) | 233GB |
Read the bottom row first. The most aggressively compressed build anyone has published is still more than seven RTX 5090s’ worth of memory.
Short answer: not on graphics cards you can put in a desktop. Mistral Large 4 runs locally only on a machine with 256GB or more of system or unified memory, at 1- or 2-bit quality and slow speeds — or on a rented multi-GPU server node. For a home GPU, run a model that was built to fit one.
Rent an 8-GPU node on RunPodWhat Mistral released
Mistral opened the API preview on October 6, 2026 and published the weights on October 29. The model card and config.json describe:
- About 1 trillion parameters in total, with 49B active per token (52B if you count the embedding and output layers, which is the figure in the repository name).
- A mixture-of-experts design: 72 layers, the first dense and the other 71 with 128 routed experts, 4 of them active per token plus one shared expert.
- A 1M-token context window and a Pixtral-style vision encoder, so it reads images as well as text.
- Two official checkpoints: FP8 (the 1.06TB one above) and NVFP4, which Mistral says targets Blackwell datacenter GPUs (B200, B300, GB200, GB300).
On licensing, the model card names the Mistral Ecosystem Open-Weight License, a modified MIT licence, and says running the model as a hosted service needs a separate commercial licence from Mistral. Running it for yourself does not.
Who this is for
You saw “open weights” in the announcement and want to know whether the hardware you own, or could buy, runs it. If you only need Large 4’s answers rather than custody of its weights, Mistral’s own API is the cheaper route and this page will mostly confirm that.
Why the active-parameter count does not save you
Only 49B parameters do work on each token, which sounds like a 70B-class memory bill. It is not. The router can send the next token to any of the 128 experts in any layer, so every expert must be loaded and ready. MoE cuts the compute per token. It does not cut the memory needed to hold the model. That is the same trap that made Kimi K3 look runnable for about a day.
The arithmetic closes: a trillion parameters at FP8 (one byte each) is about 1TB, matching the 1.06TB checkpoint. At roughly 2 bits per weight it is about 250-300GB, matching the IQ2 builds.
The tiers, honestly
| Hardware | Memory | Smallest build that fits | Verdict |
|---|---|---|---|
| RTX 5090 | 32GB | none | about 14% of the 1-bit build |
| 4x RTX 5090 | 128GB | none | about 55% of the 1-bit build |
| Workstation, 256GB RAM | 256GB | UD-IQ1_S (233GB) | runs, slowly, at 1-bit quality |
| Workstation or unified-memory machine, 384-512GB | 384-512GB | UD-IQ2_XXS to UD-Q2_K_XL | Unsloth’s suggested tier for 2-bit |
| 8x H100 80GB node | 640GB | up to UD-Q4_K_XL | not the native FP8 checkpoint |
| 8x H200 141GB node | 1,128GB | FP8 checkpoint | fits, little left for long context |
| Blackwell node (B200 class) | 192GB per GPU | NVFP4 checkpoint | Mistral’s own SGLang example uses 4 GPUs |
The memory tiers for the GGUF builds come from Unsloth’s guidance on its repository; the server rows follow from Mistral’s examples, which use a tensor-parallel size of 8 for FP8.
How fast is “slowly”?
On a CPU-and-RAM build, every token has to read the active weights out of memory once. At about 2 bits, 49B active parameters is roughly 12-13GB per token. Desktop dual-channel DDR5-6000 moves at most about 96GB/s, which caps generation near 7 tokens per second in theory, and real numbers land below the cap. An 8-channel server board raises the ceiling about threefold. These are bandwidth ceilings, not measurements — see our methodology for how we model speed.
Seven tokens per second is readable for chat. It is painful for coding agents, which generate and re-read thousands of tokens per step.
What to run on a home GPU instead
Two recent models were designed for single-GPU memory budgets, and both are better uses of the money than chasing Large 4 at 1-bit:
- Qwen 3.8 27B: a dense model whose default Ollama Q4_K_M build is 18GB, so it needs a 24GB card. See best GPU for Qwen 3.8.
- Nemotron 3.5 Bolt 74B-A9B: NVIDIA packed a 74B MoE into a 22GB file, with vision. NVIDIA’s own recipes start at 32GB cards. See best GPU for Nemotron 3.5 Bolt.
If it is specifically a Mistral model you want locally, the smaller Mistral family still fits consumer cards — the Mistral GPU guide covers those.
Best Local Alternative
NVIDIA GeForce RTX 5090
32GB GDDR7
32GB runs Qwen 3.8 27B with long context and meets NVIDIA's recipe for Nemotron 3.5 Bolt — the realistic ceiling for one desktop GPU.
Affiliate links. We may earn a commission at no extra cost to you. Amazon ships worldwide; Shopee SG covers Singapore and ASEAN.
Which GPU should you buy?
- You want Large 4’s quality, not its weights: no GPU. Use Mistral’s API.
- You need the weights under your own control: rent an 8-GPU H200 node, or a Blackwell node for the NVFP4 build. Our cloud GPU guide compares providers, and RTX 5090 vs H100 explains why datacenter cards are a separate class.
- You want to try it at home anyway: the deciding spec is 256GB+ of RAM, not the graphics card. A GPU only helps by holding the shared and dense layers.
- You want the best model one desktop GPU can run: RTX 5090 with Qwen 3.8 27B or Nemotron 3.5 Bolt; a 24GB RTX 4090 for Qwen 3.8.
Common mistakes to avoid
- Treating “open weights” as “runs on my PC”. The licence lets you run it. The 1.06TB checkpoint decides whether you can.
- Sizing hardware from the 52B active figure. Every expert stays resident; size from the total.
- Buying four GPUs to chase it. Four RTX 5090s hold about half the smallest build. RAM capacity, not GPU count, is what makes the 1-bit build load.
- Judging the model on a 1-bit build. The IQ1 file exists so the model can load at all. Its quality is not what Mistral’s published results describe.
Final verdict
| Your situation | Best path |
|---|---|
| Want the answers | Mistral API |
| Need weight custody | Rented 8x H200 (FP8) or Blackwell node (NVFP4) |
| Want to experiment at home | 256GB+ RAM workstation, 1- or 2-bit GGUF |
| Want a strong model on one GPU | RTX 5090 with Qwen 3.8 27B or Nemotron 3.5 Bolt |
Best Budget
NVIDIA GeForce RTX 3090
24GB GDDR6X
A used 24GB card is the cheapest way to run Qwen 3.8 27B at Q4, a dense 27B that fits comfortably in 24GB.
Affiliate links. We may earn a commission at no extra cost to you. Amazon ships worldwide; Shopee SG covers Singapore and ASEAN.
Mistral Large 4 is a server model with downloadable weights. Rent the node when you need custody, use the API when you need answers, and spend your GPU budget on a model built to fit one card.
Frequently asked questions
How much memory does Mistral Large 4 need?
The official FP8 checkpoint is about 1.06TB. The smallest community GGUF build, a 1-bit quantization from Unsloth, is about 233GB and is meant for machines with 256GB of RAM. Two-bit builds run from roughly 300GB to 380GB. No single consumer graphics card comes close.
Can I run Mistral Large 4 on an RTX 5090 or 4090?
Not on its own. Even the smallest build is several times larger than a 32GB card, so the model would have to live in system RAM with the GPU helping on a few layers. A home machine needs 256GB or more of RAM to load it at all, and speed is then limited by memory bandwidth to a few tokens per second.
What should I run locally instead of Mistral Large 4?
A model built for one GPU. Qwen 3.8 27B fits a 24GB card at Q4, and NVIDIA’s Nemotron 3.5 Bolt packs a 74B mixture-of-experts model into about 22GB, with official recipes starting at 32GB cards. Both are far more practical than Mistral Large 4 at 1-bit quality.