Kimi K3: Inside Moonshot's 2.8T-Parameter Open Model, and What It Means for Fine-Tuning

Every few months the "largest open-source model" title changes hands, and most of the time the delta is incremental: a few hundred billion more parameters, a marginally better benchmark chart, the same architecture underneath. Kimi K3 is not that. Moonshot AI's newest release is a 2.8-trillion-parameter model, the first open-source model to cross the 3-trillion-parameter class, and it gets there on a genuinely new attention mechanism rather than just more of the same MoE recipe scaled up. If you're already fine-tuning open-weight models, this one is worth understanding before the weights land on July 27, 2026.

What Kimi K3 actually is

Kimi K3 is Moonshot's flagship model, built on two architectural changes layered together: Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals (AttnRes), a technique designed to help information propagate more cleanly through long sequences and deep networks. On top of that sits a heavily sparse Mixture-of-Experts setup, the Stable LatentMoE framework, which activates only 16 of 896 experts per token. That's an unusually aggressive sparsity ratio, and it's the main reason a 2.8T-parameter model is trainable and servable at all rather than being a research curiosity.

The headline numbers:

2.8 trillion total parameters, first open-source model in the 3T-parameter class

16 of 896 experts active per token, Stable LatentMoE sparsity

1M-token context window, native, not a retrofit

Native visual understanding, multimodal from the base model, not a bolt-on vision adapter

~2.5x the scaling efficiency of Kimi K2, Moonshot's own figure for compute-to-capability conversion

Full weights by July 27, 2026, Moonshot is currently aligning technical details with inference partners and open-source maintainers before the release

That last point matters for anyone planning around this: K3 is announced and its architecture is public, but the full weight release is still pending as of this writing. Moonshot has held the open-source model scale lead in 9 of the last 12 months (July 2025 to July 2026), and K3 is the continuation of that streak, not a one-off flex.

Why the attention mechanism is the actual story

Standard transformer attention scales quadratically with sequence length, which is the reason long-context models have historically been either slow, expensive, or both. Kimi Delta Attention is a hybrid linear attention design, meaning it blends the quality of full attention with the near-linear cost profile of linear attention variants. Paired with Attention Residuals, which help preserve information flow across both longer sequences and deeper stacks of layers, the combination is what makes a genuine 1M-token context window practical at this parameter count rather than a marketing number that degrades badly past 100K tokens in practice.

This is worth separating from the parameter count headline, because parameter count alone doesn't predict usefulness. A 2.8T model with weak long-context behavior would be a worse fine-tuning base for document-heavy or agentic workloads than a smaller model with genuinely stable attention over long inputs. The architecture choice here is specifically aimed at long-horizon tasks, sustained coding sessions, large codebases, multi-step agentic workflows, not just topping a leaderboard.

What it's built for: coding and knowledge work

Moonshot is positioning K3 around two capability clusters:

Long-horizon coding. With minimal human supervision, K3 is designed to sustain long-running engineering tasks: understanding and working across large codebases, coordinating terminal tools, and, notably, combining software engineering with visual reasoning. That last part means using screenshots and visual feedback as part of the workflow, which is directly useful for frontend engineering, game development, and CAD-adjacent work where "does this look right" is part of the task. Knowledge work. Beyond public benchmarks, Moonshot reports consistent gains in internal evaluations built from recurring patterns in real user-agent collaboration workflows, the kind of production-shaped tasks that don't always show up in academic benchmark suites. The claim is broad improvement across agentic knowledge-work capabilities, not just narrow benchmark wins.

Both of these map cleanly onto why teams fine-tune models in the first place: not to build a general chatbot, but to build a specialist that understands your codebase, your support history, your document corpus, or your internal tools better than a general model can out of the box.

Where K3 fits next to the models you can fine-tune today

K3 isn't available for fine-tuning yet, the weights aren't out. But it's useful to place it against models already in production fine-tuning catalogs, including InfoPlatform.ai's, where Kimi K2.6 has been available on the PRO plan for weeks:

Moonshot's own framing is instructive here: K3 isn't a replacement that obsoletes K2.6 for every use case, it's a scale-up aimed specifically at top-tier long-horizon reasoning and coding. Teams with workloads that already run well on K2.6 or GLM 5.2 don't need to wait on K3; teams pushing into genuinely long-context, agentic, or visual-plus-code territory are the ones who should be watching the July 27 release closely.

The fine-tuning question: full fine-tune vs. adapter, at 2.8T parameters

A model at this scale sharpens a question that already matters for smaller open-weight models: do you fully fine-tune, or fine-tune an adapter (LoRA-style) on top of frozen base weights? At 2.8T total parameters, even with only 16 of 896 experts active per token, the cost, VRAM, and iteration-speed gap between full fine-tuning and adapter-based fine-tuning becomes enormous. Practically, expect K3 fine-tuning (once weights ship) to be adapter-first: training a small set of low-rank matrices against the frozen base rather than touching the full parameter set, the same pattern already used for K2.6 and GLM 5.2 in production fine-tuning pipelines.

That's not just a cost optimization. Adapter isolation also means a K3 fine-tune stays a small, addressable artifact rather than a monolithic multi-terabyte checkpoint, which matters for anyone thinking about data governance, deletion, or per-tenant isolation on top of a top-tier-scale base model.

What to do between now and July 27

If you're planning to fine-tune K3 on release, the useful prep work doesn't require the weights yet:

Identify the long-context or agentic workload that justifies K3 over a smaller model. If your current fine-tune on K2.6 or GLM 5.2 already hits your quality bar, K3 is not an automatic upgrade, it's a bigger, more expensive base best justified by genuinely long-context or visual-plus-code tasks.

Get your training data ready in the same clean, deduplicated JSONL format you'd use for any other open-weight fine-tune. Base model changes don't change data prep discipline.

Budget for adapter-based training, not full fine-tuning, given the parameter count, plan compute and VRAM around a LoRA-style run against a frozen 2.8T base, not full-parameter training.

Watch for Moonshot's technical report, which is expected alongside the weight release and should cover architecture, training, and evaluation detail beyond what's in the initial announcement, that's what will tell you exactly how K3 compares to K2.6 on your specific task type.

If you're tracking training and inference spend across multiple providers already, keep that visibility in place. A top-tier-scale open model is still cheaper to fine-tune and run than most closed alternatives, but it's not free, and a tool like AICosts.ai is worth having wired in if you're comparing the true cost of a K3 run against smaller open-weight alternatives across providers.

Why this matters for the "own your weights" argument

The broader story here isn't just one model release. It's that the open-source top-tier keeps closing the gap with closed models while staying open: Kimi K3 is explicitly positioned for top-tier intelligence scenarios, long-horizon coding, knowledge work, reasoning, the same category closed top-tier labs compete in, and it ships with weights you can download, fine-tune, and own outright. That's the same trend that's already made GLM 5.2, DeepSeek V4, Qwen 3.5, and Kimi K2.6 viable full replacements for closed APIs in production fine-tuning workloads on InfoPlatform.ai. K3 pushes that top-tier further up the capability ladder without changing the underlying deal: your data, your fine-tuned weights, your inference, no vendor lock-in.

Once the weights ship and Moonshot's inference-partner alignment settles, expect K3 to slot into the same OpenAI-compatible fine-tuning path already used for K2.6, upload data, fine-tune an adapter, get a production endpoint that drops into OpenCode, Cursor, or any OpenAI-SDK app with a one-line base_url swap. If you're already running K2.6 in production and want to be first in line to evaluate K3 against your actual workload the day weights land, get started on InfoPlatform.ai now so your data pipeline and evaluation harness are ready before July 27.

FAQ

When are Kimi K3's weights released?

Moonshot has committed to releasing full model weights by July 27, 2026. As of the announcement, Moonshot is working with inference partners and open-source maintainers to align technical details before launch, and a full technical report covering architecture, training, and evaluation is expected alongside the weight release.

How many parameters does Kimi K3 have?

2.8 trillion total parameters, making it the first open-source model in the 3-trillion-parameter class. It uses an extremely sparse Mixture-of-Experts design (Stable LatentMoE) that activates only 16 of 896 experts per token, which keeps inference and training cost far below what the total parameter count would suggest.

What is Kimi Delta Attention (KDA)?

KDA is a hybrid linear attention mechanism that Kimi K3 is built on, paired with a technique called Attention Residuals. Together they're designed to help information flow more effectively through long sequences and deep networks, which is what makes K3's native 1M-token context window practical rather than degraded at long lengths.

Will Kimi K3 be fine-tunable on InfoPlatform.ai?

Not yet, the weights haven't shipped. Kimi K2.6 is fine-tunable today on InfoPlatform.ai's PRO plan, and K3 is a natural next addition to the catalog once Moonshot's weight release and inference-partner alignment land on or after July 27, 2026, following the same adapter-based fine-tuning path already used for K2.6, GLM 5.2, and DeepSeek V4.

Should I wait for Kimi K3 instead of fine-tuning on Kimi K2.6 or GLM 5.2 now?

Only if your workload specifically needs top-tier-scale long-horizon reasoning, very long context, or combined code-plus-visual-reasoning tasks that current open-weight models don't handle well enough. If K2.6, GLM 5.2, Qwen 3.5, or DeepSeek V4 already meet your quality bar, there's no reason to wait, you can fine-tune and ship today, and re-evaluate against K3 once it's available.