Kimi K3: 2.8 Trillion Parameters, Four Bits at a Time
Moonshot just announced the largest open-weight model ever built. The parameter count is the headline. The serving stack is the story.
Intro
Moonshot AI released Kimi K3 precisely on July 16. The headline number is 2.8 trillion parameters, which makes it the largest open-weight model ever announced.
We spent the launch week reading almost everything published about it, and the more we read, the less the parameter count felt like the point. The point is that a 2.8T model is being served today, at Claude Sonnet prices, with a flat rate across a one million token context window.
The reasons that is possible sit exactly at the layer this newsletter cares about: attention design, expert routing, quantization format, and cache economics.
A caveat before the numbers. The technical report is not out yet. Moonshot’s launch blog states that architecture, training, and evaluation details will arrive alongside the report, and several figures below come from community analysis of the launch documentation rather than from a paper.
Where we compute something ourselves, the assumptions are stated inline and marked as ours, and there is a confidence appendix at the end.
This is a launch-week analysis, not a postmortem.
What shipped
The facts first, because launch weeks always tend to generate a lot of useless fog. The model went live on July 16 through kimi.com, the Kimi apps, Kimi Work, Kimi Code, and the Moonshot API at api.moonshot.ai under the model ID kimi-k3.
It is also listed on OpenRouter at the same rates as the direct API. What did not ship is the checkpoint: Moonshot says the full weights land by July 27, and until the files appear on Hugging Face, K3 is a hosted model you can call but not download.
Verdent’s launch guide reports the weights are coming under a modified MIT license, which would basically match the K2 line, but the license text is unconfirmed until the release itself.
The specification, per Moonshot: 2.8 trillion total parameters, a mixture of experts activating 16 of 896 experts per token, native vision input rather than a bolted-on adapter, and a context window of 1,048,576 tokens. The model reasons before every answer. At launch, reasoning effort is locked to max, with lower effort modes promised in later updates. That detail matters more than it sounds, and we will get to it in the pricing section.
The timing is not subtle either. VentureBeat notes the release landed just a few days before the World Artificial Intelligence Conference in Shanghai, and frames it as a comeback move for a company whose position had eroded badly during DeepSeek’s rise.
Moonshot is backed by Alibaba, which also builds the Qwen line, so the Chinese open-model field is now crowded with players who share an investor and compete anyway. Moonshot’s own blog claims that for nine of the past twelve months, Kimi models have held the frontier of open-source scale.
Whatever you think of that framing, the release cadence behind it is real: K2 in July 2025 at one trillion total parameters, K2 Thinking in November with quantization-aware INT4, and now K3 at nearly three times the K2 total.
The shape of the model
Three important architectural pieces define K3: the sparsity of the expert layer, the hybrid attention stack, and a modified residual path. Each one is a serving decision as much as a modeling decision.
My advice is to start with sparsity, because the trend line is the clearest signal of where frontier MoE design is going. K2 activated 8 of 384 experts, roughly 32B active parameters out of 1T total, an activation ratio around 3.1 percent. K3 activates 16 of 896.
Moonshot has not published the active parameter count; the community model card analysis on Hugging Face estimates roughly 50B active equivalent, which against 2.8T total is circa 1.8 percent. Total capacity nearly tripled while active compute grew by maybe half.
That is the whole game: parameters are cheap to store and expensive to move, so you scale what sits in memory and hold nearly flat what has to cross the datapath every token.
DeepSeek formalized this direction at 671B total and 5.5 percent active; Moonshot is now running it harder than anyone with an open checkpoint.
The lineage is worth one sentence: the sparse expert layer goes back to Shazeer’s 2017 outrageously large networks paper, was made trainable at datacenter scale by GShard and Switch, and was pushed toward fine-grained experts with a shared trunk by the DeepSeek line; K3 extends that same axis to 896, and the references at the end walk the chain to the primary sources.
The expert layer runs on what Moonshot calls Stable LatentMoE. According to the launch documentation, routing happens in a latent space rather than directly on token representations, load is managed by a mechanism called Quantile Balancing, and overflow tokens are soft-dropped rather than hard-bounced.
All three claims obviously await the technical report, but the intent is legible: at 896 experts, classical routing instability and load skew are the failure modes that kill both training and serving, and everything named here is aimed at them.
If you have ever watched a MoE deployment where two hot experts saturate their devices while the other few hundred idle, you know why a lab would lead with the word Stable.
The attention stack is a hybrid, and it has a paper trail. K3 is built on Kimi Delta Attention, KDA, which Moonshot introduced with the Kimi Linear report in late 2025.
KDA is essentially a gated refinement of DeltaNet: a delta-rule state update with fine-grained per-channel decay instead of a scalar forget gate, implemented with chunked parallel kernels so the recurrence trains at practical speed.
In the Kimi Linear configuration these layers were interleaved with full attention at a three to one ratio, the global layers used multi-head latent attention, and the paper reported on the order of a 75 percent KV cache reduction and up to a reported 6x decode throughput gain at million-token lengths against a full-attention baseline.
K3’s global layers use Gated MLA, a gated variant of the same latent attention. Moonshot has not confirmed K3’s layer ratio, but the lineage tells you the design logic: KDA layers carry a fixed-size recurrent state that costs the same at position one million as at position one hundred, and only the global layers accumulate position-indexed KV, compressed into latents at that.
The million-token window is not priced flat out of generosity. It is priced flat because most of the stack does not pay for length.
For readers who want the update rule, KDA in the Kimi Linear formulation maintains a matrix-valued state S, rewritten each step as
S_t = (I - beta_t k_t k_t^T) Diag(alpha_t) S_{t-1} + beta_t k_t v_t^Tread right to left: decay the previous state channel by channel through Diag(alpha_t), erase the stale association along the incoming key direction, then write the new key-value pair at strength beta_t.
The per-channel diagonal gate is the refinement over Gated DeltaNet’s scalar decay, restoring some of the selectivity full attention gets for free, and the chunked kernel that makes the recurrence trainable at scale, a specialized diagonal-plus-low-rank formulation, is the same computational shape the prefill discussion below returns to.
The third piece is Attention Residuals, AttnRes, described as a drop-in replacement for standard residual connections. Instead of every layer adding onto one uniformly accumulated stream, each layer can selectively retrieve representations from arbitrary earlier layers.
The claimed motivation is depth: in very deep MoE stacks, different experts fire at different depths, and a selective residual path keeps early information reachable without forcing it through every intermediate transformation.
It is the kind of change that sounds small and touches everything, and it is the single item on this list we most want to see ablated in the report.
Around the edges: a custom activation Moonshot calls SiTU, a sigmoid tanh unit replacing the usual SwiGLU family, and a per-head variant of the Muon optimizer, scheduling learning rates at the level of individual attention heads.
Muon is a K2-era inheritance; per-head scheduling is new. Moonshot’s aggregate claim is that the architectural and data changes together give K3 roughly 2.5 times the scaling efficiency of K2, meaning more capability per unit of training compute. That number is vendor-measured, marked as unverified, and should be treated accordingly.
The vision path deserves more than a scope note, because it is half the modality story and it touches everything above. What is known: vision is native rather than adapter-based per the community model card reading, continuing the multimodal line Moonshot opened with K2.5, which means image tokens enter the same transformer stack and the same 896-expert router as text, and the multimodal scores ship on day one rather than in a point release.
The API mechanics are documented even where the economics are not: images go in as base64 payloads or platform file references, public image URLs are not accepted, and OpenRouter lists image input at the standard rates.
The agent framing leans on vision explicitly, a model pitched for iterating against screenshots, logs, and runtime feedback rather than captioning, and Moonshot’s own multimodal numbers, 81.6 on MMMU-Pro and 94.3 on MathVision, sit in the launch tables.
What is not known at all is the exchange rate. The rate card is a single line, and whether an image bills at a multiplier over the $3 text rate or simply lands as however many tokens the tokenizer emits is unconfirmed, a gap the pricing trackers have flagged since day one.
The serving consequences do not wait for the answer. Whatever the patch geometry turns out to be, images are prefill load: a screenshot-heavy agent loop is a long-prompt workload wearing a different costume, and it collides with prefix caching in a way text does not, because a refreshed screenshot mid-transcript invalidates every cached token downstream of it.
The append-only discipline from the harness section applies doubly to media: attach new frames at the tail, never edit them in place.
And there is a broad research question waiting in the checkpoint. Early fusion means text and vision tokens share the router and the expert pool.
Whether the 896 experts partition by modality, some effectively becoming vision specialists, or blend, is exactly the kind of question the July 27 weights let anyone answer with an afternoon of routing statistics, and the answer feeds straight back into pruning, because a modality-partitioned pool compresses very differently from a blended one.
Four bits from the start
The decision that makes 2.8T servable is not in the attention stack. It’s in the number format. K3 was trained with quantization-aware training from the supervised fine-tuning stage onward: MXFP4 for weights, MXFP8 for activations.
This is not a post-training quantization pass applied to a bf16 checkpoint. The model learned to live inside four-bit weights, compensating for quantization error during training instead of absorbing it afterward.
Terms, briefly, because the acronym is doing a lot of work. MXFP4 is the four-bit member of the OCP Microscaling family: weights are grouped into blocks of 32, each element is an E2M1 float, one sign bit, two exponent bits, one mantissa bit, and every block shares a single 8-bit power-of-two scale.
The element grid has eight magnitudes, 0, 0.5, 1, 1.5, 2, 3, 4, 6, spaced logarithmically rather than uniformly. Activations ride one tier up as MXFP8, eight-bit elements under the same block-scale scheme, where the precision matters for numerical stability.
The migration from K2 Thinking, which shipped INT4 quantization-aware training in November 2025, to floating-point four bits here is a change of grid, and grids have consequences you can measure. So we measured, on a synthetic but realistic weight distribution, a Gaussian with and without a sprinkle of 30-sigma outliers, block size 32, both formats implemented per spec:
# MXFP4 (E2M1 elements, shared E8M0 block scale) vs INT4 groupwise, block size 32.
# Question: what does each grid do to a realistic weight distribution?
import numpy as np
rng = np.random.default_rng(7)
E2M1 = np.array([0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0]) # magnitudes
def quant_mxfp4(w, block=32):
out = np.empty_like(w)
for i in range(0, w.size, block):
b = w[i:i+block]
amax = np.abs(b).max()
if amax == 0:
out[i:i+block] = 0; continue
shared_exp = np.floor(np.log2(amax)) - 2 # emax of E2M1 is 2
scale = 2.0 ** shared_exp # E8M0: power of two only
x = np.clip(np.abs(b) / scale, 0, 6.0)
idx = np.abs(x[:, None] - E2M1[None, :]).argmin(1) # round to nearest level
out[i:i+block] = np.sign(b) * E2M1[idx] * scale
return out
def quant_int4(w, block=32):
out = np.empty_like(w)
for i in range(0, w.size, block):
b = w[i:i+block]
amax = np.abs(b).max()
if amax == 0:
out[i:i+block] = 0; continue
scale = amax / 7.0 # fp16 scale, uniform grid
out[i:i+block] = np.clip(np.round(b / scale), -7, 7) * scale
return out
def report(name, w):
for label, q in (("mxfp4", quant_mxfp4(w)), ("int4-g32", quant_int4(w))):
rel = np.linalg.norm(w - q) / np.linalg.norm(w)
small = np.abs(w) < np.quantile(np.abs(w), 0.99) # error on the 99% mass
rel_small = np.linalg.norm(w[small] - q[small]) / np.linalg.norm(w[small])
dead = (q[small] == 0).mean() * 100 # small weights crushed to zero
print(f"{name:18s} {label:9s} relRMSE {rel:.4f} relRMSE(99% mass) {rel_small:.4f} zeroed {dead:5.2f}%")
w_gauss = rng.normal(0, 0.02, 1 << 20)
report("gaussian", w_gauss)
w_out = w_gauss.copy() # 0.2% outliers at 30x sigma
hit = rng.choice(w_out.size, w_out.size // 500, replace=False)
w_out[hit] = rng.choice([-1, 1], hit.size) * 0.6
report("gaussian+outliers", w_out)
Executed output:
gaussian mxfp4 relRMSE 0.1141 relRMSE(99% mass) 0.1102 zeroed 8.19%
gaussian int4-g32 relRMSE 0.0970 relRMSE(99% mass) 0.1012 zeroed 13.37%
gaussian+outliers mxfp4 relRMSE 0.1919 relRMSE(99% mass) 0.2353 zeroed 13.01%
gaussian+outliers int4-g32 relRMSE 0.1499 relRMSE(99% mass) 0.2585 zeroed 18.40%
Read the table honestly and neither format dominates.
INT4 with a floating-point scale per block really wins global reconstruction error in both regimes, because its fifteen uniform levels and exact scale beat eight logarithmic levels under a power-of-two scale on Gaussian mass.
What the FP4 grid buys is essentially at the bottom of the distribution: it zeroes meaningfully fewer small weights, 8.2 versus 13.4 percent on clean data, 13.0 versus 18.4 with outliers, and it carries the bulk of the mass with lower error when an outlier inflates a block’s scale, because the log-spaced levels keep resolution near zero exactly where a uniform grid goes coarse.
Small weights are where accumulated damage shows up at scale, so this is not a cosmetic difference, but it is also not the headline reason for the format.
The headline reasons are that the exponent-only scale is free to apply in hardware, that Blackwell tensor cores execute block-scaled MXFP4 natively at full rate, with the Hugging Face community analysis reporting the same for AMD’s MI400 class, and that quantization-aware training exists precisely to claw back whatever the grid loses.
A lab that wanted a research artifact would release bf16 and let the community fight over quants. A lab that wants its model actually served releases the four-bit weights it trained, in the format the current accelerator generation runs at full rate.
The arithmetic consequence: 2.8 trillion parameters at four bits is 1.4 TB of element payload, and closer to 1.5 TB once the per-block scales ride along, since a 32-element block carries 136 bits, 4.25 effective bits per weight.
The same model in FP16 would be about 5.6 TB. That factor of nearly four compounds through the memory system: fewer devices to hold the model, and a quarter of the bandwidth per token spent reading weights. Be precise about what the four does and does not buy on the wire, though.
Expert dispatch and combine move activations, and activations are MXFP8, so all-to-all traffic halves relative to a bf16 model rather than quartering. The full 4x applies to weight movement: initial loading, host offload, expert migration and rebalancing.
Weights and activations shrink by different factors, and conflating them is how serving estimates go wrong by 2x.
There is a subtler consequence for the open release. When the weights drop on July 27, they will be born four-bit.
There will be no ambiguity about which community quant is faithful, because the FP4 checkpoint is the reference, not a lossy derivative. The flip side is that the fine-tuning story gets strange: LoRA and QLoRA on top of an already-quantized MoE base is thinly explored territory, a point the Hugging Face overview raises as an open research question.
The first serious K3 fine-tunes will be experiments in method, not just in data. One loose thread already dangles here. One launch tracker describes OpenRouter’s route as Moonshot’s hosted INT4 endpoint.
That may be nothing more than four-bit shorthand for the MXFP4 checkpoint, or it may mean the serving fleet runs a different quantization than the training format, and the difference matters to anyone who plans to benchmark API behavior against the weights they download. File it with the questions the report owes.
What it takes to serve it
Napkin arithmetic first, ours and labeled as such, all of it downstream of the ~50B active estimate.
A single decode token’s forward pass touches 16 experts per MoE layer plus the shared trunk, on the order of 25 GB of weight reads at four bits. Against the roughly 8 TB/s of HBM bandwidth on a current Blackwell-class part, that is about three milliseconds of pure weight traffic, an upper bound near 320 tokens per second even if one device could hold the whole model, which it cannot.
The ratio is the point of the exercise: at FP16 the same pass would read 100 GB per token and the model would be bandwidth-strangled on any realistic hardware. Four-bit weights are not an optimization here. They are the enabling condition.
That 25 GB figure is also a batch-size-one number, and batch size one is the worst case this architecture has. Here is the mechanism, because it is the actual thesis of extreme sparsity. Dense-layer weights are read once per forward pass and amortize cleanly across every token in the batch.
Expert weights amortize only when tokens share experts, and with 16 of 896 routing, sharing takes scale.
Rather than assert the curve, we simulated it, sixteen trials per point, with two popularity models: uniform routing, which is what Quantile Balancing is trying to buy, and a moderately skewed distribution with a coefficient of variation around 0.7, which is what real routers produce when you let them:
# How expert weight traffic per token falls with batch size, and what routing
# skew does to it. E=896, k=16, ~1.5 GB per expert at MXFP4 (assumption, tier E).
import numpy as np
rng = np.random.default_rng(3)
E, K, GB_PER_EXPERT, TRIALS = 896, 16, 1.5, 16
batches = [1, 8, 64, 256, 1024, 4096]
def run(pop): # pop: expert popularity distribution, sums to 1
uniq_gb, imb = [], []
for B in batches:
u, m = [], []
for _ in range(TRIALS):
draws = np.concatenate([rng.choice(E, K, replace=False, p=pop)
for _ in range(B)])
counts = np.bincount(draws, minlength=E)
u.append((counts > 0).sum())
m.append(counts.max() / (B * K / E)) # max load over mean load
uniq_gb.append(np.mean(u) * GB_PER_EXPERT / B)
imb.append(np.mean(m))
return uniq_gb, imb
uniform = np.full(E, 1 / E)
skewed = rng.dirichlet(np.full(E, 2.0)) # cv ~0.7, mild hotness
g_u, i_u = run(uniform)
g_s, i_s = run(skewed)
print(f"{'batch':>6} {'uniform GB/tok':>15} {'imbalance':>10} {'skewed GB/tok':>14} {'imbalance':>10}")
for j, B in enumerate(batches):
print(f"{B:>6} {g_u[j]:>15.2f} {i_u[j]:>10.1f} {g_s[j]:>14.2f} {i_s[j]:>10.1f}")
print(f"floor at large batch: {E * GB_PER_EXPERT:.0f} GB / B")
print("EP sizing at MXFP4: EP64 ->", E // 64, "experts/GPU,", E // 64 * GB_PER_EXPERT,
"GB | EP128 ->", E // 128, "experts/GPU,", E // 128 * GB_PER_EXPERT, "GB")
Executed output:
batch uniform GB/tok imbalance skewed GB/tok imbalance
1 24.00 56.0 24.00 56.0
8 22.72 14.9 21.82 18.8
64 14.42 4.9 12.65 7.4
256 5.20 2.7 4.80 5.5
1024 1.31 1.8 1.30 4.6
4096 0.33 1.4 0.33 4.6
floor at large batch: 1344 GB / B
EP sizing at MXFP4: EP64 -> 14 experts/GPU, 21.0 GB | EP128 -> 7 experts/GPU, 10.5 GB
Three things in that table are worth internalizing. First, the amortization cliff: per-token expert traffic barely moves from batch 1 to batch 8, because eight tokens drawing 128 expert slots land on about 120 distinct experts, which is to say almost no sharing at all.
It falls to 5 GB per token at batch 256, where the batch has touched nearly the whole 1,344 GB expert pool once, and approaches the 1,344 divided by B floor beyond a thousand concurrent tokens.
Second, the imbalance column, max expert load over mean, is the one your step time actually obeys, because the slowest device sets the pace of every synchronous layer.
Under uniform routing it decays toward 1.4 as the law of large numbers kicks in. Under skew it plateaus at about 4.6 and never recovers: the hot expert stays hot, no matter how big the batch gets. Third, notice the trade hiding between the columns: skew slightly reduces traffic, popular experts get reused, while wrecking balance.
Quantile Balancing, read through this table, is Moonshot choosing the uniform column, paying the worst-case traffic to buy the converging imbalance, because a 4.6x hot spot at expert parallelism width is a 4.6x tax on every token.
The structural conclusion stands either way: a 1.8 percent activation ratio is only cheap at large batch under wide expert parallelism, where every device holds a slice of the pool, reads its residents once per step, and serves whichever tokens the all-to-all delivers.
At EP64 that slice is 14 experts and 21 GB per GPU; at EP128, 7 experts and 10.5 GB, numbers that fit comfortably beside a trunk shard and a KV allocation.
Extreme sparsity does not just permit big-batch serving. It demands it, which is why models shaped like this favor operators with deep pools of concurrent traffic and punish anyone trying to run them hot for three users.
The all-to-all itself can be sized on the back of the same napkin. Per MoE layer, each token’s hidden state ships to its 16 expert devices and 16 partial outputs ship back: roughly 2 x k x d_model bytes at FP8.
K2’s hidden width was 7,168 and K3’s is unpublished, so treat this as parameterized: 2 x 16 x 7,168 is about 229 KB per token per MoE layer, double K2’s 8-way routing at the same width.
Across an assumed 60 MoE layers that is roughly 14 MB of fabric traffic per generated token, and at 15,000 tokens per second of aggregate throughput the expert-parallel group is moving on the order of 200 GB/s. Inside an NVLink domain at 1.8 TB/s per GPU that is background noise.
Stretched across racks on 400G links at 50 GB/s each, it is the budget. We have written before about the all-to-all tax and the rack-scale networking built to pay it; K3 is precisely the class of model that hardware exists for.
If the 2 x k x d term feels abstract, here is the entire MoE data path, route, dispatch, grouped GEMM, combine, in one page of numpy, checked against a dense reference:
# The whole MoE data path in one page: route, dispatch, grouped GEMM, combine.
# Toy sizes so it prints; the shapes are the ones a serving engine juggles.
import numpy as np
rng = np.random.default_rng(0)
T, D, E, K = 8, 16, 32, 4 # tokens, hidden, experts, top-k
x = rng.normal(size=(T, D)).astype(np.float32)
W = rng.normal(size=(E, D, D)).astype(np.float32) * 0.1 # one matrix per expert
router = rng.normal(size=(D, E)).astype(np.float32)
logits = x @ router # [T, E]
topk = np.argsort(-logits, axis=1)[:, :K] # [T, K] expert ids
gates = np.exp(logits[np.arange(T)[:, None], topk])
gates /= gates.sum(1, keepdims=True) # softmax over the k winners
# dispatch: replicate each token k times, then sort rows by destination expert
flat_expert = topk.ravel() # [T*K]
flat_token = np.repeat(np.arange(T), K) # [T*K]
order = np.argsort(flat_expert, kind="stable") # the permutation
xp = x[flat_token[order]] # [T*K, D] permuted buffer
# grouped GEMM: one segment per expert, ragged sizes
counts = np.bincount(flat_expert, minlength=E)
offsets = np.concatenate([[0], np.cumsum(counts)])
yp = np.empty_like(xp)
for e in range(E): # in CUDA: one grouped kernel
s, t = offsets[e], offsets[e + 1]
if t > s:
yp[s:t] = xp[s:t] @ W[e]
# combine: unpermute and gate-weighted sum back to [T, D]
y = np.zeros_like(x)
np.add.at(y, flat_token[order], yp * gates.ravel()[order][:, None])
# reference: dense loop over tokens and their experts
y_ref = np.zeros_like(x)
for t in range(T):
for j in range(K):
y_ref[t] += gates[t, j] * (x[t] @ W[topk[t, j]])
print("matches dense loop:", np.allclose(y, y_ref, atol=1e-5))
print("permuted buffer rows:", xp.shape[0], f"({K}x the token count)")
bytes_per_tok = 2 * K * D * xp.itemsize
print(f"wire bytes per token, this layer: 2 x k x d x {xp.itemsize} = {bytes_per_tok}")
print("tokens per expert:", counts.tolist())
Executed output:
matches dense loop: True
permuted buffer rows: 32 (4x the token count)
wire bytes per token, this layer: 2 x k x d x 4 = 512
tokens per expert: [2, 0, 1, 1, 0, 2, 0, 0, 2, 0, 1, 1, 1, 2, 0, 1, 2, 1, 1, 1, 0, 2, 0, 0, 2, 0, 0, 2, 2, 3, 1, 1]
Every serving-engine complication is already visible in the toy. The permuted buffer holds k times the token count, so top-16 doubles the activation memory and permutation traffic of a top-8 model before a single expert FLOP is spent.
The per-expert segments are ragged, 0 to 3 tokens here, thousands in production, which is why the expert matmul is a grouped GEMM with runtime-sized groups rather than a clean batched one. And the scatter-add in the combine is the reduction the fabric has to carry back.
Scale the token count by six orders of magnitude and the expert count by 28, and the sort, the offsets, and the segment loop become the router kernel, the dispatch layout, and the grouped GEMM that the serving stacks will spend the next quarter optimizing.
Memory next. Weights at ~1.49 TB fit inside a single 8x B200 node, 1,536 GB, with about 50 GB to spare, and that sentence is a trap: the spare must hold every KV block, activation buffer, routing table, and graph on the box, so a one-node K3 is a demo, not a deployment.
Moonshot’s own launch guidance recommends a supernode of at least 64 accelerators, which community sizing reads as eight nodes of eight 80 GB GPUs, around 5 TB aggregate, a floor with headroom for cache and parallelism.
On a GB200 NVL72 the weights occupy roughly a tenth of the rack’s 13.4 TB of HBM. The DeepSeek 671B generation already made rack-scale expert parallelism a normal conversation; K3 raises the resident-byte requirement by roughly 4x…. and makes the full NVL72 look like the natural unit for a single replica rather than an extravagance.
The long-context economics deserve their own bytes, because the flat 1M pricing rests on them. In the DeepSeek V3 configuration that MLA descends from, each token stores a compressed latent per global layer: a 512-wide KV latent plus a 64-wide decoupled positional key, 576 elements, 576 bytes at FP8.
If K3 carries something like the Kimi Linear ratio, call it 15 to 20 global layers out of the stack, a full million-token sequence holds roughly 9 to 12 GB of KV, ours again and resting on the assumed ratio. A comparable dense model, 60 layers of grouped-query attention with 8 KV heads of dimension 128 at FP16, holds about 246 KB per token, 246 GB at the same window, per sequence.
That is a factor of about 25, and it is the difference between a million-token session being a scheduling problem and being a physical impossibility. The KDA layers contribute a fixed-size state regardless of length, which is a rounding error next to either number.
This is the architecture showing through the rate card: Moonshot can price the window flat because most of the model does not pay for it.
Prefill is the other half, and it is why the serving system is disaggregated. Filling the full window on a cache miss costs $3.15 of input at the sticker rate, $0.31 on a hit, and the compute behind the miss is brutal.
Linear-layer FLOPs alone run about 2 x 50B x 1M tokens, on the order of 10^17, several seconds on a full H100 node at perfect efficiency and the better part of half a minute at realistic utilization, before the attention term, which grows quadratically on the global layers and dominates at this length even confined to a quarter of the stack.
The KDA layers prefill differently again: a chunked scan, parallel within each chunk with recurrent state carried across chunk boundaries, a kernel shape with nothing in common with FlashAttention.
Prefill at 1M is a compute event; decode is a bandwidth event; the two want different hardware shapes and different batching. Moonshot serves K3 on Mooncake, its KVCache-centric disaggregated architecture, which splits prefill and decode across separate node pools and treats the cache as a first-class pooled resource between them.
Mooncake is not opaque vendor infrastructure: the design won the best paper award at FAST 25, the code is open-sourced, and its transfer engine has been integrated into the mainstream serving stacks, including vLLM and SGLang.
The company reports cache hit rates above 90 percent on coding workloads. That figure is vendor-reported, but it is plausible for agent traffic, where the same system prompt, repository context, and conversation prefix recur on every loop iteration, and the published architecture is engineered to produce exactly that number.
The hit rate is what makes the pricing model work, which brings us to the bill by way of the software that has to exist first.
Extreme sparsity does not just permit big-batch serving. It demands it.
What has to change in the serving stacks
Self-hosting K3 is not a config file away, and it is worth being specific about where the engineering actually lands, because the July 27 story will be written in pull requests.
Start at the router. Every MoE layer produces logits of shape tokens by 896, takes a top-16, and must land tokens in per-expert segments fast enough not to shadow the GEMMs.
The grouped GEMM behind it now has up to 896 runtime-sized groups per layer per rank, and the tokens-per-expert variance the toy above showed becomes tile quantization waste at CUDA scale: an expert with 37 tokens still occupies whole tensor-core tiles.
The mature answers are persistent grouped kernels that pull ragged work from a queue instead of launching per group, and the CUTLASS grouped GEMM machinery most stacks already wrap.
What changes for K3 is scale, 896 groups against the 256 the current kernels were tuned around, and precision, because the groups now multiply block-scaled FP4 weights against FP8 activations.
Precision is where the hardware generations split. Blackwell’s fifth-generation tensor cores execute block-scaled MXFP4 natively, scales applied in the MMA itself, so B200-class serving runs the checkpoint as stored.
Hopper has no FP4 MMA path at all: an H100 deployment dequantizes weights to FP8 or BF16 in registers on the way into the matmul, the Marlin and Machete family of mixed-input kernels, which exist and are fast for dense weight-only GEMMs but are immature for 896-group grouped MoE at launch.
That dequant tax is not cosmetic. It is compute spent on format conversion inside the innermost loop, and it lands directly on the self-hosting ledger below: the 37 tokens per second per GPU that beats the sticker price assumes the grouped W4A8 path works well on Hopper, and today that is an assumption.
The communication layer has a shelf to raid but not a finished product. DeepSeek open-sourced DeepEP, NVSHMEM-based dispatch and combine kernels with FP8 payloads, IBGDA for the internode path, and a low-latency mode for decode, and it is the obvious starting point.
It was also built and tuned around the 256-expert, top-8 shape of the V3 generation. K3 doubles the per-token payload with top-16 and multiplies the destination fan-out with 896 experts, so the kernel launch geometry, the SM budget reserved for communication, and the overlap schedule that hides all-to-all latency behind expert compute all need retuning.
The dual-microbatch overlap trick, one microbatch’s communication hidden under another’s GEMMs, matters more here, not less, because the fabric term we sized above grows with k while the compute term holds near flat.
The subtlest problem is the cache manager, and it is the one we would watch first. Serving frameworks assume a homogeneous per-layer paged KV cache; the hybrid allocators built for the Mamba-generation models already broke that assumption, and K3 stresses it further: paged 576-byte latents for the global layers next to fixed-size recurrent state for the KDA layers.
Decode is straightforward. Prefix caching is not, because the entire cache-hit economy assumes you can resume from a stored prefix, and for a recurrent layer that means storing the state at the reuse boundary, not just the tokens.
A KV pool like Mooncake’s holds paged latents naturally; checkpointed KDA states at chunk boundaries are a second object type with different lifetime and size semantics, and how Moonshot handles state-restore for the linear layers is, to us, the single most interesting undisclosed detail in the serving stack.
Whoever implements it in the open stacks will be making a real design decision, not just a port.
The good news is how much is already on the shelf. The chunked delta-rule kernels for KDA shipped in the open with the Kimi Linear release through the flash-linear-attention line.
FlashMLA covers the global layers on Hopper. DeepEP covers the fabric, pending retuning. Mooncake’s transfer engine is merged where it needs to be.
What does not exist yet, anywhere public, is the integration: one engine that schedules block-scaled grouped GEMMs, hybrid attention with two kernel families, a two-type cache, and disaggregated state transfer, for a 1.5 TB checkpoint.
That is the artifact to watch for in the weeks after July 27, and its arrival date, more than the weights themselves, decides when open K3 is real.
The bill
The rate card, from Moonshot’s platform pages:
All figures per million tokens. Web search tool calls bill separately at $0.004 each, though the tool ships flagged as being updated and one guide advises against production use for now. Vision input pricing is unconfirmed, per the vision section above.
The first thing the sticker tells you is positioning. K2 built its reputation as the cheap frontier model. K3 abandons that identity: at $3 and $15 it matches Claude Sonnet 5 to the cent, undercuts Opus 4.8 at $5 and $25 and GPT-5.6 Sol at $5 and $30, and costs several times its own K2.6 sibling.
Within the Chinese open cohort it is the expensive option, with DeepSeek V4 Pro at roughly a sixth of its input price and GLM 5.2 around half. Moonshot is telling you it believes K3 competes on capability, and it has priced away the discount narrative to prove it.
The second thing is that the sticker is not the effective price. The cache tier is. At a 90 percent hit rate, effective input cost is 0.9 times $0.30 plus 0.1 times $3.00, about $0.57 per million tokens.
Agent workloads with stable prefixes live near that floor. Caching is automatic, with no cache ID or TTL management, which removes the usual engineering excuse for missing it, and OpenRouter’s traffic view says the same thing from the demand side: average realized K3 prices run 60 to 80 percent below list once caching is counted.
Two more levers are worth knowing. K3 ships with no batch tier, the 60 percent batch discount on Moonshot’s menu covers only the K2 line, and the API defaults max_completion_tokens to 131,072, configurable up to the full window, a cap worth leaving in place until you have watched what max-effort thinking does to a bill.
The third thing cuts the other way. K3 always reasons, at max effort, and reasoning tokens bill as output at $15. Artificial Analysis measured 130 million output tokens to run its Intelligence Index evaluation, against a median near 63 million for comparable reasoning models.
The same measurements put decode speed at 62 tokens per second, under the 72.7 median for the price tier, and split latency into the two numbers an always-thinking model needs: about 2 seconds to the first streamed token, 34.24 seconds to the first answer token, which at the measured decode rate is roughly 2,000 thinking tokens spent before the answer begins on a standard 10,000-token workload.
Running the full index cost $2,709.75, nearly all of it output. Independent testing has recorded over thirteen thousand thinking tokens on a single short task, roughly twenty cents of deliberation before the first answer token.
Artificial Analysis puts the blended cost at $2.31 per million tokens on a 7:2:1 cache, input, output mix, but your mix will vary, and a maximally verbose reasoner shifts the mix toward the expensive column. “Price it on your own traces” is easy to say obv, so here is the whole model in twenty lines, run against three workload archetypes at launch rates:
# Request-level cost model for kimi-k3 at launch pricing.
# $3/MTok input on miss, $0.30 on cache hit, $15/MTok output.
# Thinking is always on and bills as output.
IN_MISS, IN_HIT, OUT = 3.00, 0.30, 15.00
def request_cost(in_tokens, cached_frac, answer_tokens, thinking_tokens):
inp = in_tokens * ((1 - cached_frac) * IN_MISS + cached_frac * IN_HIT)
out = (answer_tokens + thinking_tokens) * OUT
return inp / 1e6, out / 1e6
archetypes = [
# name, input, cached share, answer, thinking, requests
("short chat turn", 2_000, 0.20, 300, 13_000, 1),
("agent loop, 40 iters", 80_000, 0.95, 500, 6_000, 40),
("1M-doc analysis", 900_000, 0.00, 2_000, 10_000, 1),
]
print(f"{'workload':<22}{'$ input':>9}{'$ output':>9}{'$ total':>9}{'output share':>14}")
for name, i, c, a, th, n in archetypes:
ci, co = request_cost(i, c, a, th)
ci, co = ci * n, co * n
print(f"{name:<22}{ci:>9.3f}{co:>9.3f}{ci+co:>9.2f}{co/(ci+co):>13.0%}")
Executed output:
workload $ input $ output $ total output share
short chat turn 0.005 0.200 0.20 98%
agent loop, 40 iters 1.392 3.900 5.29 74%
1M-doc analysis 2.700 0.180 2.88 6%
The distribution of pain is the finding. On a small chat turn, deliberation is 98 percent of the bill: the answer costs half a cent and the thinking costs twenty.
A forty-iteration agent loop, the workload K3 is aimed at, still spends three of every four dollars on output even with a 95 percent cache hit rate doing everything right on the input side. Only the cold million-token document flips the ratio, and that shape pays $2.70 of prefill precisely once, after which it becomes an agent loop too.
The lever ordering falls out directly: for two of the three archetypes that matter, the promised low and high effort modes are worth more than any cache optimization you can do, and until they ship, the K2.6 rate card at $0.95 and $4 remains the right tool for every request that does not need a frontier model deliberating at full depth.
The self-hosting ledger
The weight release invites an obvious question:
at what point does running K3 yourself beat paying Moonshot?
The napkin again, ours. Take the community floor configuration, 64 H100-class GPUs, and a rental price of $2 per GPU-hour: $128 per hour for the cluster.
To generate tokens cheaper than the $15 output rate, the cluster must sustain about 2,370 tokens per second aggregate, or 37 per GPU. That is an achievable number for a well-tuned MoE deployment with real batch depth, with the caveat the kernel section earned: on Hopper it assumes the grouped W4A8 dequant path matures, because every register spent converting FP4 is throughput the break-even does not get.
To beat the $2.31 blended rate that cached API traffic actually pays, the same cluster must sustain about 15,400 tokens per second, 240 per GPU, which is a very hard number for a model this size on Hopper-generation hardware.
Independent color corroborates the memory wall from an unexpected direction: an early community stress test found a 1.5 TB unified-memory Mac Studio cluster at the edge of what a K3-class million-token stack can carry, which is the same wall the 1.49 TB storage figure predicts.
The comparison is loose by construction: the API’s $15 covers output alone while your cluster serves whole requests, the blended target moves with your cache profile, and $2 per H100-hour is a spot figure that swings with region and term.
The asymmetry survives all of that. Self-hosting K3 competes with Moonshot’s sticker prices and loses badly to Moonshot’s cache economics, because the cache discount is not a margin decision you can copy, it is a serving architecture, Mooncake’s pooled KV plus a workload with 90 percent prefix reuse, that a single-tenant cluster cannot replicate without the traffic to feed it.
The teams for whom self-hosting pencils out are the ones who need it for reasons the rate card does not price: data residency (the hosted API processes in Singapore, per an EU procurement review), fine-tuned variants, latency floors, or sovereignty.
For everyone else, the July 27 download is leverage in a pricing negotiation, not a cost reduction.
Benchmarks, with the usual caution
Moonshot’s own evaluation places K3 at frontier level but behind the two strongest closed models, Claude Fable 5 and GPT-5.6 Sol, an admission worth respecting because labs rarely volunteer it.
On GDPval-AA v2, the aggregate knowledge-work evaluation, the reported ordering is Fable 5 Max at 1815, GPT-5.6 Sol Max at 1747.8, K3 at 1687, and Opus 4.8 at 1600. On the science and multimodal side, Moonshot reports 93.5 on GPQA-Diamond, 81.6 on MMMU-Pro, and 94.3 on MathVision.
The coding numbers are where it gets interesting. K3 posts 88.3 on Terminal-Bench 2.1, half a point behind GPT-5.6 Sol, and takes the top score outright on Program Bench at 77.8 and on SWE Marathon at 42.0. SWE Marathon is the long-horizon one, and a model leading there while merely placing elsewhere is consistent with the design: a million-token window that holds an entire repository plus the full history of a long session is worth more on sustained work than on sprints.
FrontierSWE lands at 81.2, against a reported 86.6 for Fable 5 on the same benchmark, and DeepSWE at 67.5.
On agentic evaluations, VentureBeat’s read of the launch documentation has K3 first in four of eight real-world automation benchmarks, including Automation Bench, SpreadsheetBench 2, and BrowseComp, and second to Fable 5 in most of the rest.
The reported scores include 91.2 on BrowseComp, a 95.0 F1 on DeepSearchQA, and 30.8 on Automation Bench, which tells you as much about Automation Bench as about the model.
The claim we find most interesting is methodological: Moonshot says these results came from a single-agent setup running on the raw million-token window, with no context compression or management scaffolding (BrowseComp reads 90.4 in that configuration against the 91.2 headline figure, both vendor numbers).
Taken at face value, that is a real data point in the context-versus-orchestration argument, raw window plus strong retrieval beating elaborate multi-agent plumbing, and it needs independent replication before it graduates from press release to argument.
Two launch-week numbers deserve a somewhat explicit skepticism. K3 jumped from rank 18 to rank 1 on LMArena’s Frontend Code Arena within hours, at a score of 1679.
The placement itself has real sourcing, the Associated Press reported K3 at the top of the frontend ranking, but arena scores during a hype cycle measure attention as much as ability; check back in a month.
BenchLM’s weighted leaderboard still lacked a K3 row at press time, an absence worth noting, but the first independent aggregate has landed: Artificial Analysis scores K3 at 57 on its Intelligence Index, fourth of 189 models tracked, against an average of 31 for comparable models, and the index rolls up nine evaluations including GDPval-AA v2, Terminal-Bench 2.1, GPQA Diamond, and Humanity’s Last Exam.
Frontier company, by an outside ruler. The 2.5x scaling efficiency figure, the 90 percent cache hit rate, and the single-agent protocol are all vendor-reported. None of this means the numbers are wrong. It means the confirmations are scheduled for after July 27.
One demo is worth relaying because it is verifiable in kind if not yet in fact. Moonshot reports that K3, in a single 48-hour autonomous run, designed a serving chip for a nano model on its own architecture using open-source EDA tools against the Nangate 45nm library: four square millimeters, timing closed at 100 MHz, 1.46 million standard cells, 0.277 MB of SRAM, an INT4 multiply-accumulate array with fused dequantization, and a simulated decode throughput above 8,700 tokens per second. A staged demo, obviously, and simulation is not silicon.
But as a choice of demo it is telling: the lab wanted to show long-horizon agency on exactly the kind of task this audience does for a living.
A second staged demo aims at researchers: to reproduce the I-Love-Q universal relations in computational astrophysics, K3 reportedly cross-validated more than twenty papers, evaluated over three hundred equations of state, caught inconsistencies in published formulas, and shipped three thousand lines of Python plus an interactive dashboard in about two hours, against Moonshot’s estimate of one to two weeks for an experienced researcher.
Vendor-staged, like the chip, and chosen with the same message: long horizons are the product.
Where it strains
Moonshot names three weaknesses, and all three have operational consequences.
The first is thinking history sensitivity: agent harnesses that truncate or rewrite the model’s reasoning traces degrade quality significantly. Launch coverage adds the reason: K3 was trained in a preserved thinking history mode, so a harness that drops the thinking, or a session handed mid-flight from another model, is operating outside the training distribution.
For agent builders this is a real constraint, and it compounds with the cache economics, because the two failure modes share a root cause: mutating the transcript.
Strip the thinking and quality drops; touch anything upstream in the serialized prefix and you lose the $0.30 tier and re-prefill from the perturbation point.
The defense against both is the same discipline, an append-only history with a verifiable prefix, cheap enough to enforce in a dozen lines:
# K3 penalizes two things at once: perturbing the prompt prefix (you lose the
# $0.30 cache tier) and truncating its reasoning traces (quality degrades).
# Same defense for both: an append-only history with a verifiable prefix.
import hashlib, json
def fingerprint(messages):
blob = json.dumps(messages, sort_keys=True, separators=(",", ":"))
return hashlib.sha256(blob.encode()).hexdigest()[:16]
def prefix_intact(old, new):
return len(new) >= len(old) and fingerprint(new[:len(old)]) == fingerprint(old)
history = [
{"role": "system", "content": "You are a code review agent."},
{"role": "user", "content": "Review PR 4217."},
{"role": "assistant", "content": "<thinking>...</thinking> Two issues found."},
]
appended = history + [{"role": "user", "content": "Fix the first one."}]
truncated = [history[0], history[1],
{"role": "assistant", "content": "Two issues found."}] # thinking stripped
reordered = [history[1], history[0], history[2]] # system moved
for name, h in (("append-only", appended), ("thinking stripped", truncated),
("system reordered", reordered)):
print(f"{name:<20} prefix intact: {prefix_intact(history, h)}")
# Client shape (not executed here; the endpoint is OpenAI-compatible):
# client = OpenAI(base_url="https://api.moonshot.ai/v1", api_key=KEY)
# r = client.chat.completions.create(model="kimi-k3", messages=history)
# history.append({"role": "assistant", "content": r.choices[0].message.content})
Executed output:
append-only prefix intact: True
thinking stripped prefix intact: False
system reordered prefix intact: False
The two False rows are the two expensive mistakes. A harness that “helpfully” compacts old reasoning fails the quality constraint and the cache constraint in the same commit; one that rebuilds the message list per call, reordering tools or regenerating timestamps, silently pays full prefill price on every iteration.
Fingerprint the prefix in development, assert it in production, and design summarization as a new branch rather than an edit to history. On long sessions where the window genuinely fills, the append-only rule forces the honest version of the tradeoff: fork the conversation with an explicit summary and eat one cold prefill, rather than quietly degrading the model mid-branch.
The second weakness is overeagerness. In ambiguous situations K3 tends to act rather than ask. On benchmarks that rewards decisiveness. In production it is how an agent cheerfully migrates the wrong database. The mitigation is boring and necessary: explicit confirmation gates in the harness, because the model will not supply them.
The third is polish. Moonshot concedes the subjective experience trails Fable 5 and GPT-5.6 Sol even where benchmark numbers are close. Benchmark parity and product parity remain different finish lines.
What July 27 decides
The weight release is the actual event; July 16 was the trailer.
The license text comes first. Modified MIT is reported across the launch coverage and would match the K2 family precedent, and the precedent has teeth worth knowing: the modified MIT that ships with K2.7 Code adds an attribution requirement that triggers at 100 million monthly active users or 20 million dollars of monthly revenue.
If K3 inherits the clause it is a display obligation for giants rather than a commercial restriction, but the binding text is the LICENSE file that lands with the weights, and nobody should sign a deployment plan before reading it.
Day-one inference support is the second gate, and the stacks section above is the checklist: block-scaled grouped GEMM at 896 groups, retuned dispatch and combine at top-16, the two-type cache with KDA state restore, and a Hopper W4A8 path that does not bleed the break-even dry.
Moonshot says it is working with inference partners ahead of the drop, and the Mooncake integrations already sitting in those codebases suggest the relationship is real; the commits will tell.
Third, reproduction. Someone outside Moonshot needs to run the benchmark suite, and the single-agent long-context protocol specifically.
Fourth, the research surface the checkpoint opens. With 896 experts in the open, expert specialization can finally be probed at frontier scale. Pruning experiments will ask whether the pool compresses to something that fits a smaller cluster, and what the quality curve looks like on the way down.
AttnRes can be ablated at small scale to see whether it generalizes or only pays at depth. And someone will attempt the first LoRA on a natively FP4 base and discover what breaks.
Then, the field is not holding still. The DeepSeek V4 line is already live at a fraction of K3’s price, and community reporting has a V4 refresh in staged testing; GLM 5.5 and MiniMax Pro are both expected around the trillion-parameter mark, and Qwen 4 is on the horizon. Whatever lead K3 holds is measured in weeks, and DeepSeek’s answer is the one to watch.
Our read, calmly: K3 does not win the frontier. By Moonshot’s own tables it sits a step behind Fable 5 and Sol. What it breaks is the assumption that open weights trail the frontier by six months and a capability class.
The gap is now single-digit percentage points and a dated download link, at Sonnet prices, on an architecture whose every major choice, the sparsity ratio, the hybrid attention, the four-bit training, the disaggregated serving, was made with the inference bill in mind. The link goes live on July 27, and that is where the work starts.
Sources and confidence
A, primary: Moonshot’s K3 launch blog and platform documentation: release dates, architecture component names, context window, pricing, stated limitations, chip design demo, benchmark tables. Published background used for parameterized math and the serving-stack analysis: the OCP Microscaling format specification, the Kimi Linear technical report (KDA design, the three to one hybrid ratio, KV reduction and decode speedup figures, the open kernel release), the Mooncake paper and open-source release (FAST 25 best paper; transfer engine integrated in mainstream serving stacks), the DeepSeek V3 MLA configuration (512 plus 64 latent dimensions), and the DeepEP communication library.
B, independent measurement: Artificial Analysis: Intelligence Index score of 57 (fourth of 189), output token consumption on the index run (130M vs a 63M median), decode speed of 62 tokens per second against a 72.7 tier median, latency of about 2 seconds to first token and 34.24 seconds to first answer token, evaluation cost of $2,709.75, and blended cost ($2.31/M at 7:2:1). OpenRouter: listing at matching rates, and platform-reported realized prices 60 to 80 percent below list under caching.
C, informed secondary: VentureBeat on benchmark placements, release timing, and corporate context. The Hugging Face community model overview for the ~50B active parameter estimate, MXFP4 and MXFP8 details, and deployment sizing. Verdent and multiple pricing guides for the comparative rate card and the modified MIT report.
D, vendor-claimed, unverified: the 2.5x scaling efficiency over K2, the 90 percent cache hit rate, the single-agent no-compression benchmark protocol, launch-week arena rankings, and the chip demo results.
E, our arithmetic and code: the batch amortization simulation, the quantization grid experiment, the dispatch reference implementation, the cost model, the prefix fingerprint demo, all-to-all sizing, KV cache bytes, prefill FLOPs, and the self-host break-even. Every code block in this issue was executed as printed, CPython 3.12.3 with numpy 2.4.4, with fixed seeds; outputs are shown verbatim. Assumptions are stated inline; every figure downstream of the ~50B active estimate, the assumed 7,168 hidden width, the ~60 MoE layer count, the ~1.5 GB per expert, and the assumed global-layer ratio inherits their uncertainty and will be recomputed when the technical report publishes the real dimensions.
The Kimi K3 technical report had not been published at the time of writing. Numbers in this issue may be revised when it appears, and we will follow up after the July 27 weight release.
The “fact check” ledger
Every load-bearing claim, its source tier, and how it was checked. Rows marked recomputed were re-derived at build time by the scripts shipped with this issue.
References
Primary and background sources. arXiv identifiers for the Kimi Linear and Gated DeltaNet entries were verified against arXiv during fact-checking; the remaining identifiers are standard citations for their papers.
Moonshot AI. Kimi K3 Tech Blog: Open Frontier Intelligence. kimi.com/blog/kimi-k3, July 2026.
Moonshot AI. Kimi K3 quickstart and platform pricing. platform.kimi.ai, July 2026.
Kimi Team. Kimi Linear: An Expressive, Efficient Attention Architecture. arXiv:2510.26692, 2025.
Yang, S. et al. Gated Delta Networks: Improving Mamba2 with Delta Rule. arXiv:2412.06464, 2024.
Yang, S. et al. Parallelizing Linear Transformers with the Delta Rule over Sequence Length. arXiv:2406.06484, 2024.
DeepSeek-AI. DeepSeek-V3 Technical Report. arXiv:2412.19437, 2024.
DeepSeek-AI. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. arXiv:2405.04434, 2024.
Qin, R. et al. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving. arXiv:2407.00079; USENIX FAST 2025, best paper.
Rouhani, B. et al. Microscaling Data Formats for Deep Learning. arXiv:2310.10537, 2023. See also the OCP Microscaling Formats Specification v1.0.
Micikevicius, P. et al. FP8 Formats for Deep Learning. arXiv:2209.05433, 2022.
Shazeer, N. et al. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. arXiv:1701.06538, 2017.
Lepikhin, D. et al. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding. arXiv:2006.16668, 2020.
Fedus, W. et al. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv:2101.03961, 2021.
Wang, L. et al. Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts. arXiv:2408.15664, 2024.
Kimi Team. Kimi K2: Open Agentic Intelligence. arXiv:2507.20534, 2025.
Liu, J. et al. Muon is Scalable for LLM Training. arXiv:2502.16982, 2025.
Ainslie, J. et al. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. arXiv:2305.13245, 2023.
Kwon, W. et al. Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv:2309.06180, 2023.
Zheng, L. et al. SGLang: Efficient Execution of Structured Language Model Programs. arXiv:2312.07104, 2023.
Open-source repositories referenced: MoonshotAI/Kimi-Linear, fla-org/flash-linear-attention, deepseek-ai/DeepEP, deepseek-ai/FlashMLA, kvcache-ai/Mooncake, on GitHub.
Launch-week coverage and trackers cited in text: VentureBeat, the Hugging Face community overview, Artificial Analysis, OpenRouter, BenchLM, and the pricing and readiness guides from Verdent, eesel, aireiter, kie.ai, avenchat, wan27.org, NxCode, digitalapplied, and TECHi, all July 2026.
Parameter counts are cheap headlines. The real engineering revolution lives in the memory bus. ⚡
Moonshot's Kimi K3 hits 2.8 trillion parameters, but it only activates 16 out of 896 experts per token. That is a 1.8% activation ratio (~50B active). Tripling capacity while holding active compute nearly flat is how you survive the HBM bandwidth wall.
Here is what actually makes 1M token context servable at Sonnet prices:
1. Native MXFP4 weights with MXFP8 activations trained straight from SFT. No lossy post-training quant hacks. E2M1 micro-scaling blocks of 32 elements hold zero-end weights intact even when 30-sigma outliers hit. 📊
2. Hybrid KDA attention. Fixed-size matrix states update via channel-decay delta rules. Length does not explode the KV cache because 75% of the stack does not pay for sequence depth. 🧠
3. Disaggregated Mooncake serving. Prefill compute lives on separate nodes from decode bandwidth, hitting 90% cache reuse.
There is a massive trap hiding in your agent harnesses. K3 was trained with preserved reasoning history. The moment your framework truncates or 'helpfully' summarizes past thinking traces, you do two fatal things. You knock the model out of its training distribution, degrading intelligence. You break the serialized prefix hash, dropping off the $0.30 cache tier back onto the $3.00 cold prefill penalty. 💸
If you modify the transcript mid-loop, your agent converts software errors straight into vendor API revenue. Append-only history is no longer a stylistic preference. It is a hard microarchitectural constraint.
Are you enforcing cryptographic prefix immutability in your local agent harnesses, or are you quietly re-paying full prefill taxes on every tool turn? 🤔
(⊙_⊙)