Kimi K3.1 Pro: Where Moonshot's Scores and Independent Measurements Disagree, and Why the Open Weights Lose Cyber Data

Moonshot AI released Kimi K3.1 Pro on October 23, 2026. It is Kimi K3, released in July, with much more reinforcement learning on top. It has a 1-million-token context window, and Moonshot says the full weights will be published by November 4. In Artificial Analysis's independent testing, K3.1 Pro beat Claude Opus 5.5 by 7.6 points on Terminal-Bench 4.0, a terminal coding benchmark. On a knowledge-accuracy index it trailed both Opus 5.5 and GPT-6 Astra by a wide margin. The fairer description is a model that is strong at terminal and agent work and weaker on knowledge questions, not an "open-weights Opus." The weights Moonshot plans to publish come from a training run with cybersecurity data removed.

This post covers what changed from K3 and where you can use it, Moonshot's table and the independent numbers on the same axis, what the cyber data removal means for the open weights, developer reaction, and where the model fits for developers in Korea.

What changed from K3 and where to get it

Moonshot describes K3.1 Pro as a post-trained version of K3. The base model is the same; the company scaled up post-training compute and ran joint reinforcement learning across hundreds of domains. Moonshot calls it a "3T-class" open-weight model. Alongside it, Moonshot released Kimi K3.1 Pro Highspeed, which it says runs 2–3× faster.

These channels were live on launch day:

On the API, thinking is always on. reasoning_effort accepts low, high and max, and the default is max. As a flagship model it unlocks after a top-up of at least $1, and Zero Data Retention is available to API customers. In Kimi Enterprise, administrators have to enable it; access is off by default at launch.

For existing users, the bigger change may be the retirement of the previous generation. The Kimi API pricing page says K2-series models will not be offered to newly registered users from October 31, 2026, and that all K2-series models are planned to be retired by the end of December 2026. Services calling K2.6 or K2.7 Code have about two months to move.

Moonshot's table against independent measurements

Moonshot's tech blog includes a benchmark table comparing K3.1 Pro with Claude Opus 5.5, Claude Fable 5.1, GPT-6 Astra, DeepSeek-V4.1-Flash-1017 and GLM-5.3. The footnotes say Moonshot ran K3.1 Pro itself; for the other models the source varies by benchmark. Some rows, such as Terminal-Bench 4.0, take vendor or public-leaderboard numbers, while for others, such as OfficeQA Pro, the footnote says competing models were evaluated in Moonshot's chosen harness (Claude Code, or Codex for GPT-6 Astra). Artificial Analysis measures several of the same benchmarks independently, so we compiled the three that overlap on one axis.

Terminal-Bench 4.0 is where the sources disagree most. In Moonshot's table, K3.1 Pro scores 65.7%, 0.7 points behind Opus 5.5 at 66.4%. In Artificial Analysis's run the order flips: 67.2% for K3.1 Pro and 59.6% for Opus 5.5. Moonshot's footnote says the Opus 5.5 figure is the higher of the vendor's blog number and the official leaderboard. The score-versus-cost chart in the same blog post plots Opus 5.5 at 64.85%, so the post carries two different numbers for it.

The other two benchmarks point the same way in both sources. On Terminal-Bench-Science 0.1, which covers terminal tasks in scientific research, K3.1 Pro is more than 20 points behind Opus 5.5 in both. On Humanity's Last Exam, Opus 5.5 leads by 8 to 9 points in both. Terminal-Bench 4.0 is the only benchmark where the ranking depends on who ran it.

Artificial Analysis also publishes an Intelligence Index that combines ten evaluations. K3.1 Pro scores 55, fourth of 215 models, behind Claude Opus 5.5 (58), Claude Sonnet 5.5 (56) and Muse Wonder Full (56). Comparing it with K3 as measured by the same organization shows what moved.

Artificial Analysis labels K3 as Max and Opus 5.5 as Max (Default Fallback), and shows no effort label for K3.1 Pro. We compiled the values from each model's data, rounded to one decimal. Terminal-Bench 4.0 rose more than fivefold, from 12.6% to 67.2%, while Humanity's Last Exam gained only 5.6 points. AA-Omniscience Index rewards correct answers and penalizes wrong ones, so it measures how reliable a model's knowledge is. K3.1 Pro's 29.5 is far below Opus 5.5 (46.4) and GPT-6 Astra (43.4). The reinforcement learning appears to have lifted agent execution far more than the breadth of what the model knows.

Artificial Analysis calls the model "notably slow and very verbose." When we checked on October 25, output speed was about 58 tokens per second, and the model used 150 million output tokens to run the Intelligence Index against a median of 87 million. Listed API pricing is $2 per million input tokens and $10 per million output tokens.

Where it leads and trails in Moonshot's own table

Even in Moonshot's table, K3.1 Pro is not near the top everywhere. We picked the rows with the largest gaps. These are Moonshot's numbers and have not been reproduced by a third party yet.

It leads on long coding tasks, question answering over financial documents, and the knowledge row marked as using tools (HLE-Full). The gaps where it trails are larger. On ProgramBench, reported as a binary score where a task counts only at 95% or higher, K3.1 Pro scores less than half of Opus 5.5. On HLE-Diamond, a set of hard knowledge questions, it is 21.5 points behind. In the same table, HLE-Full with tools (70.0) puts it ahead, while the HLE-Full row without a tools label (51.1) has it behind Opus 5.5 (59.8). Search tools appear to cover gaps in what the model knows, the same direction as the AA-Omniscience result above.

Why the open weights lose cyber data

The cybersecurity section is the most unusual part of the release. Moonshot says vulnerability-research ability rose steadily as post-training scaled. Tracing the cause, it found cyber-related content in the general mid-training corpus and in the reinforcement learning environments. For the open-weight release, Moonshot says it found and removed cyber-related data from both stages while keeping general coding tasks.

Moonshot shows the effect on xdev-bench-hard, an internal benchmark that asks the model to reproduce real vulnerabilities in software such as the Linux kernel and Node.js and build working exploits. A checkpoint without the removal solved 90.5% at pass@3; the final K3.1 Pro solved 81.0%. On the external ExploitBench, K3.1 Pro scores 74.7, below Opus 5.5 (91.0) and GPT-6 Astra (100). Moonshot states a principle that its open weights should not extend the absolute frontier of cyber capability, and says it intends to raise the cyber capability of its open models step by step.

For security teams this cuts both ways. Once the weights are out, a team can audit old code on its own servers without sending it anywhere. But that model has been deliberately held back on offense, so for red-team work that needs working exploits it is unlikely to replace the top commercial models.

Developer reaction

Reaction split on Hacker News, both on the launch thread (1,066 points and 655 comments when we checked on October 25) and on a separate thread about the Artificial Analysis result.

Supporters focused on how close an open model has come to the frontier. User thepasch, citing the Artificial Analysis numbers, wrote:

"I think it's still remarkable that this is effectively an open model that matches the frontier for the first time in a long time, pretty much since DeepSeek R1." — thepasch, Hacker News

Skeptics said the Artificial Analysis ranking alone doesn't make it better than Opus 5.5:

"while this is an awesome model, I have a bridge to sell to folks who think (based on AA leaderboard) that this is better than Opus 5.5" — option, Hacker News

User mppm passed along the argument that AA-Omniscience is more representative than other metrics and wrote that by that measure K3.1 Pro is about six months behind the frontier (Hacker News). Others complained about subscription limits: johnmlussier wrote that on the $99-a-month plan, one session hit the 5-hour quota within 47 minutes (Hacker News). On the cyber data removal, one commenter asked where "cybersecurity data" ends and ordinary software knowledge begins (besterman23, Hacker News).

There is also a dispute over promotion. VentureBeat reported on October 24 that screenshots posted by an X user showed a Kimi promotional brief telling creators not to add the paid-partnership label. The article notes that the document has not been independently authenticated and that the screenshots do not show Moonshot authorized such a campaign. It found no company response to the specific allegations.

What changes for developers in Korea

For coding agents that work in a terminal, K3.1 Pro is now a real candidate alongside the commercial models. Its Terminal-Bench 4.0 lead held up in independent testing, so loops of build, test and log reading look worth trying on your own repositories. Because it was also measured as slow and verbose, batch jobs that can wait seem a better first fit than interactive tools where latency matters.

Where knowledge accuracy matters, the case for switching is weak. In legal, medical or fact-checking work, where what the model knows drives answer quality, the AA-Omniscience and HLE-Diamond gaps turn directly into risk. Moonshot's numbers suggest search tools narrow the gap, but no third party has reproduced that yet.

For teams that cannot send data out, the November 4 weights are the real change. Using the API means sending requests to a service run by Moonshot AI, a Chinese company, so Korean teams handling personal data or internal code will likely need an internal review even with Zero Data Retention. Published weights would let them run the model on a domestic cloud or their own hardware. At "3T-class" size, though, it will likely need a multi-GPU server, which looks out of reach for most small teams.

Services on the K2 series now have a deadline. When moving to K3 or K3.1 Pro before the end-of-December retirement, check that reasoning_effort defaults to max; leave it unset and you get the slowest, most token-hungry setting.

For background, we covered Opus 5.5 itself in our Claude Opus 5.5 post, and the DeepSeek model in the same comparison table in our DeepSeek V4.1-Flash-1017 post.

Limitations

What generalizes is the direction (strong on terminal work, weaker on knowledge), not the individual numbers. Harness, reasoning effort and tool access differ between sources, so part of any gap may be setup rather than model.

Sources

• Moonshot AI, Kimi K3.1 Pro tech blog: https://www.kimi.ai/blog/kimi-k3-1-pro
• Kimi API, Kimi K3.1 Pro quickstart: https://platform.kimi.ai/docs/guide/kimi-k3-1-pro-quickstart
• Kimi API, model inference pricing (K2 retirement notice): https://platform.kimi.ai/docs/pricing/chat
• Artificial Analysis, Kimi K3.1 Pro model page: https://artificialanalysis.ai/models/kimi-k3-1-pro
• Hugging Face, moonshotai organization: https://huggingface.co/moonshotai
• Hacker News, Kimi K3.1 Pro and K3.1 Pro Highspeed: https://news.ycombinator.com/item?id=50212065
• Hacker News, Kimi K3.1 scores 55 on Artificial Analysis: https://news.ycombinator.com/item?id=50213337
• VentureBeat, report on the promotion allegations: https://venturebeat.com/technology/moonshots-kimi-k3-1-pro-flirts-with-open-weights-leadership-amid-paid-influencer-scandal-allegations
• The Decoder, launch report (2026-10-23): https://the-decoder.com/chinese-ai-model-kimi-k3-1-pro-is-closing-in-on-anthropics-claude-opus-5-5/