Kimi K3 just got cheaper to serve — no new hardware required
OCT 18, 2026 · 5 MIN READ

A new open-source serving optimization cuts GPU-hours per token for Kimi K3 by 24–41% at production batch sizes. No new silicon, no new contract — just more tokens out of the lease you already have.

Most of the time, a GPU lease gets cheaper per token in only one way: you fill it with more traffic. This week there was a second way. vLLM shipped a rewritten decode path for Kimi K3 that does more work per GPU-hour on the exact same chips — no new order, no new contract, just a software update.

01 / WHAT CHANGED

On October 16, 2026, the vLLM project published a specialized decode path for Kimi K3 on NVIDIA’s Blackwell Ultra platform. The change is all software: fusing operations that used to run as separate kernels, cutting the per-layer kernel count roughly in half, and overlapping weight loading with communication instead of waiting on it. Nothing about the model changed and nothing about the hardware changed. Depending on batch size and GPU count, the same GPUs now produce 1.3× to 1.9× as many decode tokens per second.

02 / WHY THIS MATTERS MORE WHEN YOU OWN THE GPUs

If you buy tokens from a hosted provider, a speedup like this is the provider’s business, not yours — whether it reaches your invoice as a lower price is their call. If you lease the GPUs yourself, the logic flips. The lease is a fixed bill, set by the arithmetic we’ve written about before: the same amount whether the GPUs serve one request or a billion. A serving-stack improvement doesn’t touch that bill. It just means every GPU-hour you’re already paying for does more work. That is the whole case for owning the stack instead of renting the output: when inference software gets better, the gains land on your side of the ledger automatically.

03 / THE NUMBERS, AT OUR $5/GPU-HOUR TARGET

Take a 16-GPU deployment running at a realistic production batch size, somewhere between 16 and 64 concurrent requests. At $5/GPU-hour, that lease costs the same whether it’s serving the old decode path or the new one. What changes is the cost per million tokens it produces: roughly $21.61 falls to $12.64 at batch 16, and $9.98 falls to $6.16 at batch 64 — a 38–41% drop with the lease bill unchanged.

24–41% more tokens / GPU-hour

At production batch sizes (16–64 concurrent requests), the same leased GPUs and the same $5/GPU-hour rate now clear more tokens for the same bill — pure margin, not new traffic.*

The single-user number is more dramatic — vLLM reports over 840 tokens per second for one request on 16 GPUs, up from about 490 — but that configuration is the least efficient use of 16 GPUs regardless of the speedup. The number that moves your bill is the production-batch one.

04 / THE CATCH

Two honest caveats before you resize anything. First, vLLM benchmarked this on GB300 rack-scale systems, not the standalone B300 we lease — same Blackwell Ultra generation, different physical configuration, so treat these figures as directional until you benchmark your own traffic. Second, the headline single-user figure assumes a fixed speculative-decoding acceptance rate; real traffic with real prompts will land somewhere else. Decode speed also isn’t the whole bill — prompt processing isn’t covered by these figures and won’t move as much.

05 / WHAT TO DO WITH THIS

01Already leasing? Update your serving stack and benchmark it against your own traffic before you order more GPUs for growth you may already have headroom for.

02Still deciding whether to lease? Add it to the 30% breakeven math: the leased side of that comparison just got cheaper, which only pulls the breakeven utilization down further. That is the direction serving software keeps moving it.

If you’re running Kimi K3 as a leadership assistant or any other large open model on B300 capacity, request a quote and tell us your current traffic. We’ll tell you honestly whether a software update covers your next six months of growth, or whether you actually need more GPUs.

Hardware is the expensive, slow-moving part of this stack. Software is the part that keeps getting better underneath you, for free, as long as someone keeps you current.

* Benchmark figures come from vLLM's October 16, 2026 writeup on an optimized decode path for Kimi K3, checked October 18, 2026. vLLM tested on NVIDIA GB300 rack-scale systems (8 or 16 GPUs in one NVLink domain), a different physical configuration from the standalone B300 we lease; both are Blackwell Ultra generation, but results on a leased deployment will differ and should be benchmarked against your own traffic. The single-user 842 tokens/sec figure assumes a fixed speculative-decoding acceptance rate and is a best case, not a guarantee. Dollar figures use our ~$5/GPU-hour target for qualifying 36-month commitments; your quote depends on GPU type, configuration, region and term.