The weights dropped on July 27. Moonshot put Kimi K3 on Hugging Face under a Modified MIT license, the download button works, and somewhere right now a person with an RTX 4090 is watching a 594GB MXFP4 checkpoint crawl toward a 2TB SSD, convinced that llama.cpp will sort out the rest. It won’t. I want to kill this dream cleanly, with numbers, before you waste a weekend on it.
Short answer: no. Not on a 4090, not on a 5090, not on a Mac Studio, not on the 8x H100 box your startup rents. As of late July 2026, Kimi K3 is a multi-node deployment or it is an API call. There is no third option yet.
The memory math, tier by tier
Kimi K3 is a 2.8-trillion-parameter Mixture-of-Experts model with 896 experts. The native MXFP4 release is already aggressively compressed — about 594GB just to download — and the total weight footprint lands around 1.4TB once scales and runtime overhead are counted. Here’s what that means against real hardware:
| Tier | Memory you have | Kimi K3 needs | Coverage |
|---|---|---|---|
| RTX 4090 | 24GB | ~594GB (MXFP4 weights alone) | ~4% of the download |
| RTX 5090 | 32GB | ~594GB | ~5% |
| 4x RTX 5090 rig | 128GB | ~594GB | ~22% |
| Mac Studio 512GB | 512GB unified | ~594GB | short before overhead |
| 8x H100 80GB (one node) | 640GB | ~1.4TB total footprint | not enough |
| Multi-node H100/H200 cluster | 1.5TB+ | ~1.4TB | the actual floor |
Sit with those middle rows for a second. A full DGX-class node — eight H100s, the machine most people picture when they say “serious AI hardware” — cannot hold the full weights. The realistic floor is multiple interconnected nodes of H100s or H200s. That’s not a homelab. That’s a rack with a networking budget.
”But it’s MoE — only 16 experts are active!”
This is the myth doing the heavy lifting in every optimistic Reddit thread. Yes, K3 routes each token through just 16 of its 896 experts. And yes, that makes inference compute dramatically cheaper than a dense 2.8T model would be. But the router picks different experts every single token, which means every expert has to be sitting in memory, ready to fire. MoE sparsity saves FLOPs, not bytes. All 896 experts are resident, all the time.
It is worth being clear that no Moonshot model escapes this. Kimi K2 is ~1T, which puts its Q4 weights near 600GB — smaller than K3 by a factor of two or three, and still multi-node. Ollama lists kimi-k2.6, kimi-k2.7-code and kimi-k3 as cloud-only entries with no local weights to pull. K3’s native format is already 4-bit, so there is no magic quant waiting to shrink it another 95%. Days after release, no full-fidelity GGUF port has been confirmed either, so the “wait for community quants” plan currently has nothing on the calendar that changes the math.
What you can actually do (three lanes)
Lane 1 — hosted API. Moonshot’s own API, OpenRouter, Together, and Fireworks all serve K3, and day-one vLLM support meant hosted capacity spun up fast. If you want K3’s quality, this is the honest path for anyone without a cluster line item.
Lane 2 — rent the cluster. If you genuinely need K3 under your own control — fine-tuning experiments, private data, weight surgery — rent multi-GPU capacity by the hour instead of pretending a workstation will do it. Our cloud GPU for LLM guide walks the provider math; for scale context on why datacenter cards are a different species, see RTX 5090 vs H100.
Rent H100/H200 Capacity on RunPod→Lane 3 — self-host a model that fits, from a different family. This is what your money actually buys locally, and it is not a Moonshot model: the whole Kimi line is served rather than downloaded. A 24GB card runs Qwen 3 32B or Gemma 4 31B Dense at Q4 with agent-grade reliability and room for context — see the best GPU for Kimi K2 guide for why the Moonshot models themselves are out of reach. Check NVIDIA GeForce RTX 4090 on Amazon→Buy on Shopee SG→ The same logic applies to any big-model itch: running 70B on a single GPU is a solved problem; running 2.8T is not.
Compare Multi-GPU Rig Prices on Vast.ai→Common mistakes I’m already seeing
- Buying hardware for a model you’ll run twice. Even if a heroic community effort eventually squeezes some K3 variant onto a 512GB unified-memory machine, ask how often you’d load it versus a fast 32B you use daily. Novelty inference is a terrible reason to spend five figures.
- Confusing “open weights” with “runnable weights.” Modified MIT means you may run it anywhere. The 1.4TB footprint decides whether you can.
- Assuming K2 is the local fallback. It is not — at ~1T its Q4 weights are around 600GB, and Ollama carries it cloud-only like every other Kimi model. The gap between K2 and K3 is real but both sit far outside a workstation.
Verdict
As of late July 2026: use K3 through Moonshot, OpenRouter, Together, or Fireworks; rent a multi-node cluster if you truly need custody of the weights; and put your local budget into a 32B-class model on a 24GB card, because no Moonshot model fits inside a house.
Kimi K3 being open-weight is genuinely great for the ecosystem. It is not an invitation to your GPU — and neither is K2. Run a 32B at home, rent H100s when you need custody of weights, and let the API handle the rest.
Kimi K3 hardware: quick answers
How much VRAM does Kimi K3 need?
Far more than any single machine offers. The native MXFP4 download alone is roughly 594GB, and the total weight footprint with scales and overhead lands around 1.4TB. Even an 8x H100 80GB node (640GB) can’t hold the full weights, so realistic deployment means multiple interconnected H100 or H200 nodes — datacenter territory, not a workstation.
Can a quantized Kimi K3 run on consumer GPUs?
No. The release format is already 4-bit MXFP4, so the usual “wait for Q4 quants” play doesn’t apply — that compression is baked in. As of late July 2026 there are no confirmed full-fidelity GGUF ports, and even hypothetical aggressive quants would still measure in hundreds of gigabytes, more than ten times what a 32GB flagship card offers.
What’s the closest model you can actually run at home?
Not a Moonshot one — Ollama carries kimi-k2.6, kimi-k2.7-code and kimi-k3 as cloud-only entries, and K2 is itself a ~1T model needing roughly 600GB at Q4. The closest thing you can actually run is a 32B-class open model: Qwen 3 32B or Gemma 4 31B Dense, both about 20GB at Q4 on a 24GB card like the RTX 4090 or a used RTX 3090. You keep agentic capability and a permissive license at a hardware cost in the hundreds or low thousands rather than a cluster.
Frequently asked questions
How much VRAM does Kimi K3 need?
Far more than any single machine offers. The native MXFP4 download alone is roughly 594GB, and the total weight footprint with scales and overhead lands around 1.4TB. Even an 8x H100 80GB node (640GB) can’t hold the full weights, so realistic deployment means multiple interconnected H100 or H200 nodes — datacenter territory, not a workstation.
Can a quantized Kimi K3 run on consumer GPUs?
No. The release format is already 4-bit MXFP4, so the usual “wait for Q4 quants” play doesn’t apply — that compression is baked in. As of late July 2026 there are no confirmed full-fidelity GGUF ports, and even hypothetical aggressive quants would still measure in hundreds of gigabytes, more than ten times what a 32GB flagship card offers.
What’s the closest model you can actually run at home?
Not a Moonshot one — Ollama carries kimi-k2.6, kimi-k2.7-code and kimi-k3 as cloud-only entries, and K2 is itself a ~1T model needing roughly 600GB at Q4. The closest thing you can actually run is a 32B-class open model: Qwen 3 32B or Gemma 4 31B Dense, both about 20GB at Q4 on a 24GB card like the RTX 4090 or a used RTX 3090. You keep agentic capability and a permissive license at a hardware cost in the hundreds or low thousands rather than a cluster.