The inference optimization of Kimi K3 on AMD Instinct MI355X is the result published in a technical post-mortem on vLLM's official blog on October 16, 2026: under the AgentX agent workload, an eight-GPU MI355X system running vLLM delivers 24.4% more total tokens per second per GPU than an NVIDIA HGX B300, and 87.4% more tokens per dollar under the cost model used in the report. The data comes from SemiAnalysis's public InferenceX curves, with a benchmark snapshot dated October 9, 2026.
Read the comparison basis first
The comparison is set at a P90 interactivity of 50 tokens per second per user, where the MI355X serves about 7,567 tokens/s/GPU against about 6,083 for the B300. Both chips carry 288 GB of HBM3E and 8 TB/s of peak bandwidth, and the B300's peak compute is actually higher, so the gap comes from the software stack, not the spec sheet. AgentX replays real coding-agent session traces with a theoretical prefix reuse above 96% — exactly where this optimization pays off; the conclusion does not transfer directly to short, reuse-free request loads.
The optimization cuts repetition in three places
First, the cache: every GPU used to store a full copy of the MLA history; with eight-way decode context parallelism, each card now keeps one eighth, cutting the cache footprint of a 142,000-token conversation from about 1.96 GB to about 245 MB, with overflow state offloaded to host memory. Second, MoE compute: the routed experts' output projection previously ran eight times, once per GPU; splitting the coordinates means each card computes only its share and the existing all-reduce reassembles the full result. Third, kernel fusion and dead copies: 69 useless copies per decode step were removed, and several serial kernels were merged into single launches.
What it means for deployment decisions
The point of the result is not a single benchmark win, but proof that, for the same model on a different chip platform, the depth of inference-framework tuning can rewrite the cost-performance ranking. The report's GPU-hour cost model prices the MI355X at $1.50 against $2.26 for the B300, and the cost gap stacked on the throughput gap produces the 87.4% tokens-per-dollar difference; real procurement still depends on cloud pricing, supply and your own cache hit rates. For teams choosing hardware for agent services, the concrete takeaway is to measure prefix reuse on your own session traces first, then compare whole-stack throughput — not just peak numbers on a chip table.
