> For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.
I like it. If the increase in unrestricted cyber capabilities can be slowed down, please be it. The world needs to get ready, and that takes time. Way more time than even a slow progress would allow.
Unless you want to see more of what happened the past few days.
I think the argument that "cyber capabilities should be democratized, thus keep producing them now, fast, and for everybody" is a very naive one.
How will it do that without a model that's good at security? You gonna have humans manually secure the millions of lines of new AI-generated code, in addition to the existing code? Suddenly people will start doing the job they haven't been doing up to now? Not only are we not capable of auditing all the code, it would take decades to finish, and new code will be coming out at the same time. We can't catch up by hand.
The only way to fight this fire is with fire. We need AI hacking tools to find our security holes quickly and patch them. Otherwise we're just churning out more insecure software, and the only people who can defend against it are the governments who don't want to patch the holes (because they want to be able to exploit them). Without open tools, we all lose.
I am glad that at least one Chinese company seems to be listening to AI cybersecurity concerns and did concrete steps that actually reduce the problem. This was probably at least somewhat costly to them in terms of general model capabilities as well. They'd probably have gotten slightly better coding capabilities if not for removing the cyber training data and tasks.
I am glad about it as well. Don't get me wrong, I love OSS, I love personally being able to get my hands on the frontier stuff and frontier cybersec capabilities, and I dislike a few entities having exclusivity on it. Everybody does. But that absolutely doesn't mean it's the correct approach. I think people are blinded a little bit by general principles and by personal feelings, rather than operating from a point of pragmatism, consequences, and the big picture.
Also, to note: I don't claim to have the answers either. I don't even hold a strong position on this issue yet. I'm just sharing some of my intuitions, and reacting to people who, as far as I'm concerned, are making this issue all too simple.
The Pandora box can expand even further, and it matters. If it didn't matter, like you seem to be suggesting, then by the same logic there isn't a reason to push for further open-source development.
> Kneecapping cyber vulns only serve governments.
Uhh, no? Kneecapping cyber vulns server most of us. There are negatives to kneecapping it, but let's not pretend like it's perfectly okay and safe for everybody in the world to have a magic box that can hack anything and cause havoc all over the world. The more powerful, cheap, and accessible this magic box is, the worse the consequences.
This is a hard problem and there's real tension - like you said, the playing field not being leveled is a problem (major companies / entities / govts having exclusivity on the best cyber abilities etc), but everybody having access to it is a problem as well.
So let's not pretend like this is a simple problem, and don't say that it only serves the govt. That's very silly.
How does "the world" get ready, when these limits are so hard core that Opus regularly stops working when I'm just trying to find and fix security exploits in my apps?
Currently available, unhindered models can already find a bunch of exploits, not because they're smart, but because they're cheap and smart enough to just keep going for hours or days until it finds something.
The defenders, especially in the FOSS space, only have one big advantage: Unified scale. If defender A finds a vuln, then defender B doesn't need to find it again. Attackers generally act alone, so they need to do all the work themselves.
If we remove a bunch of defenders, the attackers are back at it again.
It's a bit fascinating to me how all of these labs have rolled their own (it sounds like) completely unique RL environments.
I wonder how much the industry will centralize around a few key patterns, and if we'll start to see some RL framework projects gain traction and be used by the bigger labs
> First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights.
So they admit that they could do more, but they won't?
Or it's a cop out. I could totally outrun Usain Bolt, but by design I don't push to set new World Records. I will increasingly show faster times though.
Well, in the publication they claim to have identified specific cybersecurity post training tasks that nerfed capabilities when they removed them, which seems like a reasonable methodology.
Seeing as no one besides blackhats want frontier cybercrime capability to be open source, this seems like a very sensible thing to do.
So they have published a reasonable means, and they have a motive. I'd put my money on "probably true".
The problem is, as anyone trying to address security concerns in claude can attest to, the guardrails trigger even when trying to address real bugs. Good cyber defense requires an understanding of the offense.
How you avoid the "fix my code" attack, the code being someone else's code, I do not know. But nerfing the model doesn't seem like the best path forward either.
This is not about guardrails, it's just not post-training the model on tasks that specifically relate to cyber-offense. It's a very lightweight intervention, and it makes the model less of a target for bad actors.
Please don't change "cybersecurity" to "cybercrime" from one sentence to the next as a rhetorical flourish. It comes off as disingenuous and makes it harder to get your real point across.
> Seeing as no one besides blackhats want frontier cybercrime capability to be open source
Sure. The problem is that there is no practical difference between "cybercrime capability" and "cybercrime defense capability."
All of my PRs go through an adversarial security review by GLM-5.3, because it's the only smart-enough model with a good understanding of cybersecurity issues that won't regularly refuse the job. Removing that ability from frontier models helps nobody except cyber criminals.
I'm not sure I follow. Let's say that GLM remains the frontier of open source cyber capabilities because Kimi is nerfed. Your situation, and the situation of your attackers, remain exactly the same. You run GLM to audit your code, they do the same looking for exploits.
Let us say that un-nerfed Kimi was released, and it turns out to be at the absolute frontier of cyber capabilities, Mythos 5.5 level, a step ahead of anything else. What now? Well now your situation is exactly the same as before, just that auditing and hacking got more expensive. You run a Kimi security audit, they do the same to try to pwn you. However, now the cabal of Linux kernel defenders with secret access to Mythos 5.5 all of a sudden have no advantage, where before they used to have one. And even worse now they are in a literal race to be the first ones to find every bug that Kimi is capable of finding, and patching it yesterday.
So for you, this makes no difference in practice, your capabilities are not elevated, and neither are those of the attackers. The only result of the other scenario is an overall defender disadvantage.
What your argument is missing is that I'll ship security issues that would have been caught otherwise.
You're assuming that these won't matter because attackers might not have better models than I do, but that doesn't make sense. The assumption that attackers can't find the issues because I didn't find them at release is wrong. Even if they never get better models (which they will), current models might find flaws I missed, and they also have other means of finding flaws in my software.
The best defense is to make software as secure as possible regardless of assumed attacker capabilities at the point of release.
> Seeing as no one besides blackhats want frontier cybercrime capability to be open source
I'm not interested in black hat anything, but I am very invested in making sure the products I work on are not compromised. Given that state agencies have "frontier cybercrime capability", I would like to have the same.
I wrote a few lengthier comments on some other similar replies. Summarized my argument is that it seems to me to be strictly better that select defenders have an advantage, rather than everyone being equalized. If you're not one of the privileged few that might be frustrating, but open source models upgrade the attack and defense by the same amount, so open source cyber capabilities being at the frontier only results in a defender disadvantage.
Replying here also as I'm curious about the counterargument.
I am a defender, and I really really want to have access to high quality models that are good at finding vulns and hacking. Leaving them in the hands only of government agencies and orgs with enough resources to do extensive training / fine tuning seems like the absolute worst possible case for the future security of global software systems.
Well, and I am genuinely curious here, wouldn't defender and attacker capabilities being accessible equally by both parties be objectively worse than having a limited amount of privileged defenders? Of course, you could have every piece of code go through an adversarial audit before it's pushed, but in an equal world you would have to always devote as much or more resources to this as your attackers. And historically speaking people just don't do this in practice.
In the scenario where only "verified" defenders get access to the best cyber capabilities, unverified plebian defenders and blackhats are on the same level exactly as described above, there's just also a few defenders that get to run things on godmode.
So to me, it seems that the only result of frontier cyber capabilities is removing a defender advantage. Although that advantage is not for everyone, it can only exists as long as capabilities are unequally distributed. Am I missing something?
> Well, and I am genuinely curious here, wouldn't defender and attacker capabilities being accessible equally by both parties be objectively worse than having a limited amount of privileged defenders?
Well, not really. Because those privileged defenders will never be able to predict what needs defending. So we have a few privileged "defended" places and the rest is left for privileged attackers to plunder.
Privileged attackers?? So that would be career cybersecurity experts with Project Glasswing/Daybreak/Googlewhatever access, using it to perform actual attacks? Because that's what privileged means in this context. Access to frontier models not available to the public.
Well, seeing as Claude literally snitches on everyday people planning murder and Houthis building rockets I would assume that every user in project Glasswing is heavily and very effectively monitored. I have to admit that I genuinely do not think that what you're hypothesizing is happening right now. I would bet good money that the amount of people with Glasswing access doing actual Blackhat work is currently zero.
Perhaps in the future this will change, but even if that is the case, that's still a win for the defenders. Having a time advantage, even for a few months, is still a win.
Edit: Actually, the CIA/FBI/NSA/Etc.. does probably have access and are probably trying to hack Iran and so forth, so my comment above is probably technically wrong in that respect.
I edited my comment above before realizing this is what you meant. And its a valid point I concede. Still, I disagree that advantaging this activity is the only thing these embargoes achieve. I would argue that they legitimately also give the Linux kernel, cURL and so on an advantage. And as long as that is true, this still makes it an overall win in my books, even if it also enables officially sanctioned offensive activities. I'm not a US citizen, but I would still rather get pwned by the CIA than both the CIA and Shinyhunters.
Because the people I want to defend against includes those with enough resources to train and fine tune models with strong offensive capabilities themselves.
Ah, now that's an interesting point. However, wouldn't those groups just fine tune Kimi K3.1 pro no matter if it's nerfed or not, leaving you in the same position as before?
That is, unless the nerf is easier to correct as compared to "upgrading" the models offensive capabilities. So a scenario where it is easy to "restore" offensive capabilities, but hard to "enhance" them beyond the labs best attempt. So is that the world we're living in?
The groups with a lot of resources are always going to have the best tools available. The question is what tools do I (a person with limited time and resources) get access to. I am in a better position if the delta between my tooling and their tooling is minimized.
> Or it's a cop out. I could totally outrun Usain Bolt, but by design I don't push to set new World Records.
The benchmarks show Kimi K3.1 Pro outrunning practically all models. It's weird to hear someone like you accusing them of making up excuses for not lagging behind in discussions on how they are ahead of the pack.
Not only is that not weird, it’s Occam’s Razor in action. “We could crush the competition but we are holding back” is a laughable excuse that I’m sure you wouldn’t buy in other contexts.
> "We could crush the competition but we are holding back” is a laughable excuse that I’m sure you wouldn’t buy in other contexts.
I don't think you understand. Benchmarks show just that, that they "are crushing the competition".
I mean, remember Fable? A couple of months back it was lauded as leaps and bounds above everything, that needed to be contained at all costs? Well, apparently Kimi K3.1 Pro outperforms it.
What industrial level of cope do you need to drag in order to look at the results and claim they need to make excuses about anything at all?
If we had a technique that would defeat almost every lock in the world, we might be prudent to delay releasing that until people have had time to change the locks.
Same general benefit as coordinated disclosure, except it's a big game-theory problem where we're likely to eventually have sufficiently capable defectors.
Yeah that's a huge downside. That the model does the task without tripping over its own feet, like discovering a vulnerability the model itself accidentally built 15 minutes ago and triggering its own guardrails, made the Chinese models attractive to use for security-adjacent things. Also useful to red-team your own stuff. While I can understand the reasoning behind it (and I guess having a model that's really good at both offense and defense released into the ether could lead to CCP asking some very pointed questions if Moonshot managed to get blamed for some incident stemming from their model's misuse) it's still a huge disappointment.
I don't want to give US labs the results of my DNA tests and my family tree going back to the 16th century just to secure my shitty app, but if the Chinese are bowing out as well it seems the only alternative is to admit that some script kiddie with a bunch of tokens will just pwn you now and then.
> I don't want to give US labs the results of my DNA tests
My friend, you realize that if you have had blood drawn for a genetic test, there is a very decent probability that the sequencing was done by a Chinese lab operating in the US, right? The Chinese probably have more US genomes that the US does.
Also, illumina used to ship products with hardcoded S3 access, so purchasers of used equipment could gain access to sequencing information generated by the original owners. Too many links to cover all of it, here's the Gemini summary:
If the Chinese want it, they have it. BGI does an enormous amount of sequencing. My own genome is entirely public (or was, and will be again when/if Personal Genome Project comes back) so you can watch and see if it harms me.
It's not actively nerfed like the Western frontier models, it's just downweighting offensive "cyber" as a domain for post-training/fine tuning. Which, if anything, ought to improve model capabilities wrt. other domains.
> So they admit that they could do more, but they won't?
Yeah, there's another paragraph:
> Identifying and removing cyber-related data. During development, we observed vulnerability research performance improving steadily as post-training scaled. Tracing the source of this trend, we found that cyber-related content was embedded within the general mid-training corpus, and cyber-related tasks were present in the RL environment mix — a natural byproduct of training on broad data. In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.
(emphasis mine)
It's clear that cyber is a touchy subject right now. But it's also clear that RL works on it, because it's highly verifiable. So smaller models are going to become very interesting, because they can be fine-tuned later to cover this field.
The "rule" here is, if you read the disclosure carefully, that they have cyber trained models in house. But you do not get them, you only get the non-cyber models.
It’s not much of a deterrent to a motivated organization.
They are basically saying that models themselves are already capable. If you fine tune them with some cyber security tasks, that should make them potent. :-/
There is no deterrent to a motivated and well resources organization. Post training a model in general, let alone just for offensive cybersecurity, is well within the technical and financial capabilities of many cyber criminals. There's just no way out of that.
I think this is misguided. Considering how much code is AI written these days do we really want LLMs writing insecure code because we lobotomised them?
>I hope the Chinese companies (and I say this as a patriotic American) slay these two companies and destroy their moats.
China cares about about stability way more than the US even, they aren't going to be big on AI models upsetting the governments power, and mass cyber attacks could do that.
At the end of the day we are going to have to realize AI is both dangerous and governments are going to control it. Worse if you start making and releasing your own dangerous models the 4chan party van is going to visit you.
> At the end of the day we are going to have to realize AI is both dangerous and governments are going to control it.
Your thinking is dangerous. Do you work for Anthropic?
AI is dangerous when these powers are being hobbled up by the select few. These systems threaten to put us all out of work. I refuse to be a slave to a company or a government that is taking away our livelihoods, scraping up our earnings potentials, then seeking to muzzle us and put us in jails with no rights.
I would rather burn down all of tech than have AI be owned by the elites.
Your thinking is a threat to all of our liberties.
>I would rather burn down all of tech than have AI be owned by the elites.
Then you better get your torches out now.
>I refuse to be a slave to a company or a government that is taking away our livelihoods
Don't worry, governments are fervently working on AI drones to take away your life when you get really upset about this. Oh, and they are also working at AI to detect when you write all this online, and track you down then monitor you on things like flock cameras. As they say, you'll be the first with your back against the wall when the revolution comes, I likely will be beside you for being too subversive.
>AI is dangerous when
Incorrect, AI is just dangerous. It doesn't matter who has it. The more powerful it becomes the more dangerous it gets. So, as you can tell I am an AI doomer.
----
Now, don't get me wrong, I want freedom for me, I want freedom for other people. I do not believe the technologies we're developing are conductive to having any freedom at all. Hell, if we're lucky we'll be slaves instead of all of us dead, including the select few.
>Your thinking is a threat to all of our liberties.
Lol, now this is kind of funny. "I want freedom to think whatever I want, but you, you shouldn't think like that!". Careful or you'll max out your irony meter. Anyway, "My kind of thinking" is just next ____ prediction. I see what is happening in the world around us, and based on the shape of that probability I see what is going to happen. No, you and I are not getting unlimited AI to do whatever we want. Yes, governments are going to lock down the shit out of AI for end users. Yes, governments are going to declare advanced LLMs munitions.
Balls in your court, better think of what your next move is.
Which is a great example of what makes him dangerous: with the exception of Sam Altman he's arguably the single person most responsible for the current capabilities arms race.
Anthropic has pushed the frontier harder and faster than OpenAI or anyone else in the last few years. Beginning with Anthropic's founding to quickly raise huge amounts of cash with a pitch to investors to rapidly leapfrog ChatGPT in scale, and culminating with Mythos/Fable which carved out a whole new tier of super models.
He's always talked big about Anthropic prioritising safety and their "Responsible Scaling" but his words just simply don't align with reality while they keep going bigger, faster, until only OpenAI could briefly and half-heartedly claim near parity at the top (while cutting corners in safety because the existential threat to their market position is far more pressing to any corporation than some abstract risk for humanity).
Whether it's wilful cognitive dissonance, cynical opportunism, financial greed or something else, that kind of self-assured moral superiority paired with his real world record of self-interested decision making for Anthropic's market position is a big red flag.
Sam, I'll grant you, is probably among the top 10 most dangerous, but Dario? The only thing he's done to piss people off is to predict, correctly, that releasing too-powerful models too-quickly would cause havoc and chaos. This prediction actively hurts his company! It actively harms his chances of a huge exit! It actively raises the probability that Anthropic and OpenAI get nationalized and treated in the same way as, for instance, the nuclear program!
By comparison, do you think the Chinese models are going to be trained in such a manner that you enjoy the society that results? What makes you think that?
> It's clear that cyber is a touchy subject right now.
And that's why we need models that understand "cyber" or application and data security. The offensive models are powerful, and they are in the hands of script kiddies now. The models writing our software and designing our cloud infrastructure had better understand these issues as well, just to avoid coding in vulnerabilities.
> Prohibited: high-performance detection technologies used to counter cross-border data transmission security risks, including deep-synthesis forgery detection (266401J); AI cyber offensive capability technologies, such as malicious code generation and mutation, automated mass generation and distributed large-scale deployment of anti-detection variants (266402J); large models trained using China’s Core Data (266501J); and high-performance AI agents designed for highly sensitive scenarios, such as critical information infrastructure, national defense and core government affairs (266502J).
I mean they say they will release the weights by November 4, so presumably after that it won’t be hard for people to fine-tune it to be better at cyber right?
Presumably if it knows how to find and fix bugs it can be told how to exploit them. But who knows what it will actually be capable of doing in the real world as opposed to Kiki’s benchmarks
Moonshot (Kimi) seems to be unashamedly into American culture[0], I wonder if they're paying closer attention to and giving more credence to some cultural aspects of American AI discourse including aspects of AI caution/panic.
With frontier competitors and the U.S. govt actively pushing "concerns" about Chinese models, Kimi including this statement on "offensive cyber capability" seems like smart defense. Models are opaque and training details are trade secrets, so whatever they did might be negligible or significant and no one can tell. This bit is also artfully done:
> "below what our pipeline can produce, and remains behind that of frontier proprietary models."
Like Ant and OAI, they've realized vaguely claiming to have nerfed their latest model to an unspecified degree not only looks 'good guy' prudent, it provides a built-in excuse when a model fails.
We really need some breakthrough like DiLoCo does, but for heterogeneous unstable consumer hardware to make volunteer crowdsourced AI training a reality. Yeah, I understand from current technical angle how much of a mountain that is. Call it AI@Home or whatever. I never thought it would bother me personally or even become a problem, but these companies acting like gatekeepers and virtue signaling while also in bed with governments etc.. this leads nowhere good.
I wish Opus 5.5 had an alternative just for the sake of not having a monopoly, but I have yet to see a model live up to its benchmark scores except for Fable and Opus; there is something about Anthropic that no other lab has.
Yeah, but it's not good enough. Ask Opus and Astra to make a landing page about lemon pies or whatever. Opus will get you a better result almost every time.
When the frontier is pacing itself in its speed in terms of adding new capabilities (notice how fast opus 5.5 got released after Astra), then that also means that the open weights models which were just behind are accelerating in their speed just as much as well.
Good news for us because the models which used to feel 6 months behind now feel like they are lesser.
Double threat to Anthropic and OAI because if open weights models become good enough at the current level of Sota while being cheaper (which time will tell if this model fits the bill or not), then people will use those models instead of OAI/Anthropic provided it is "good enough"
Though it has to be said that only real world use case can be trusted but I remain optimistic and hope that this turns out to be good in real world use cases as well.
But in my opinion, all of this is really good for us end users!
Extrapolating a bit, Moonshot AI provides Kimi K3 at $15 / $3 / $0.3 per million output/input/cached tokens. A 30% reduction would put K3.1 at roughly $12 / $2.4 / $0.24.
So Kimi K3.1 Pro would be generally competitive with the latest Sol and Opus models on API pricing. And that's before factoring in third-party providers, some of which serve Kimi K3 for significantly less than Moonshot. For example, inference.net offers K3 for $10.80 / $0.239 / $0.21.
(One caveat: A Claude or GPT subscription still gives you way more value compared to API pricing.)
Exciting times in any case! We really need more direct competition to OpenAI and Anthropic.
OpenAI allows you to use serviceTier=flex, dropping their price by 50%.
And not to mention the gargantuan amount of tokens you get with the US labs' subscription plans. I still cannot see a reason to use this as an end consumer.
looks extremely impressive, the full benchmark table isn't opening for some reason though so do we really have agi if small important things like that keep breaking?
It would be super useful to have more specific analyses of what areas the model is good at. I'd like to know when I should reach for this model instead of Anthropic's or OpenAI's.
For example, Opus 5.5 is better at UI design than Sol. Anyone who has used both models knows that, but it's not something you can look up anywhere.
I'm sure that there's a need for this kind of benchmarking, not just for coding.
Not sure if this is benchmarkable besides for the own person deciding which one they like more. Models difference is more about taste, preferences and prompt style.
I had a friend yesterday tell me he likes the way Claude writes and finds it easier to understand compared to Gemini. But he mostly doesn't use Claude for coding. Whereas I find Claude insufferable in writing often and prefer Gemini for writing. But I don't use Gemini for coding.
Though to be fair Opus 5.5 is better than Opus 5 was in that regard. I still prefer the OpenAI models to either of them when it comes to regular prose, though.
Furthermore, "UI design" is not a narrow enough category, I think. Sure, statistically significant differences in performance between models do manifest when benchmarking against "UI design", but there's still just so much variance within the category. Opus 5.5 can have excellent one-shot results, but totally shit the bed when it's placed in an existing codebase and should uphold the UI/UX conventions, design language, and existing component primitives already established in an app.
But that's just the obvious. There are task by task differences (component by component, page by page, context by context) that the model is worse and better at, even though they are all within the general category of UI design.
I honestly don't have the clarity of mind to explain this coherently, but the crux of my point is that there's a huge variation in it's performance in different contexts, often ones that seem near similar, and in result, even Sol can beat Opus 5.5 in some variations of UI design.
And this is my problem in benchmarking and comparing model performance in general. Presently, we try to create broad categories to benchmark against, create a set of tasks, and then average out the results. That works generally, but it's kind of misleading and it's partly what drives people to feel a disconnect between benchmarks numbers and real world usage on their specific tasks.
It's such a messy field that it's kind of maddening.
One things Kimi is useful for is accessing/querying websites that have paywalls or try to prevent use by AI agents. It is aggressive and persistent for that, though not always successful.
So, the US economy spends trillions of dollars to be at the "frontier" and China puts out a model that basically does the same thing, for free, and for a tiny fraction of the development cost. How is this sustainable?
It's not like US companies spend trillions on training itself.
Mostly it's datacenter build out and circular financing magic when the same dollars being commited multiple times by multiple companies passing hot potato to each other. They basically commit capacities 2-3 years ahead.
There would be no Chinese model without US models to distill though. So, in a way, the Chinese models also cost a trillion dollars, they just don’t cost China that much.
It's 'sustainable' because there's enough regulatory risk to using Chinese models that many will avoid them, and a 3T parameter model is still a bit beyond even many enthusiasts for running locally.
That regulatory risk argument won't work forever. It's the same with how the US bans chinese EVs which is why Tesla has high market share in the US, but in the rest of the world, chinese EVs are doing big numbers. The future is going to be full of surprises.
Maybe I'm conflating. I could be dead wrong about all this. But I'm trying to zoom out and look at the big picture. So, I'm letting the details sort of blend away and I'm just asking myself: "Is such a large financial sink in our economy, at the macro level, really going to offer the value that the investment warrants?". So, the particulars of who exactly is spending what where I guess doesn't concern me as much. What I'm thinking about is American companies, American people, American dollars going to something that seems like an iffy value proposition.
Now, if you tell me American businesses are turning cheap Chinese tokens into goods and services of high value, for Americans and people all over the world, then obviously I would have a different outlook on what's happening here.
You are conflating. The build out is almost entirely for running inference, not training. Open weight models will make that inference build out bet more likely to work as it adds diversity to the suppliers, all the data centers are not beholden to just OAI and Anthropic to buy their compute - they can host their own models (even Chinese ones, in America) if that becomes a popular option.
Zooming out, I think that build out still makes sense. There will be losers in it, but think about how ridiculous the amount of compute we're using has increased from 4 years ago. If I had to guess 60-70% of developers and engineers everywhere in the world are now burning thousands in compute individually every month, whereas it was previously approximately 0.
I see. So maybe the situation isn’t as bad as I thought. I was thinking if you took all the money and resources spent on building the biggest models, across the two big labs, the hyperscalers, the dozens of neo labs, and you reallocated that money to building value on cheaper Chinese models that would be a much better proposition than doing whatever it takes to be head of Chinese model performance by a few percentage points for a few months. And I wasn’t just referring to the cost of training one model, I was considering the total cost of maintaining an entire ecosystem with the ability to do this.
But what I’m hearing is this is a negligible cost compared to inference buildouts. Fair enough.
I've not tried it out, but in the past I've been pretty disappointed in the performance of Kimi models. Deepseek, Qwen, and GLM all appear to perform much better even though Kimi often has higher benchmark scores. Ditto for Mimo.
The older Kimi K models (2.5 and 2.6) weren't bad for the time, but all of their models have had problems with overthinking. This also makes them significantly more expensive in practice than their pricing suggests, since they produce way more output (reasoning) tokens than most other models. Anecdotally, I've also had problems with all the pre-K3 models thinking themselves in circles trying to solve very simple problems that other models didn't struggle with. (I haven't played with K3 much because it's just too expensive.)
I have the opposite impression. I find K3 to be extremely capable at full-stack work; its UI design skills are great. I'd love to use it more, but the quota on Kimi's plans is abysmal. I wonder if these new models improve that.
definitely not frontier though, as even on max both legs are in front of the frame; although I am wondering how much getting that right correlates with model intelligence.
The biggest (of many) failing of the pelican test is that it's only one prompt - what matters most these days is how well a model can drive a lengthy agentic tool calling session.
To be fair I have a feeling that if you added 'animated' to the SVG request, the output quality from many other models would improve, just because of the extra chain-of-thought work needed to fulfill the more complex request. IMHO the animation requirement should be added to the canonical pelican prompt.
If you are expecting any sort of usable limits on their subscription plan I suggest adjusting your expectations. I subscribed to $99 a month plan and within 47 minutes with one session (not asking for swarms or anything) was limited by 5-hour quota.
Welp, that ruins this news for me. I'm glad open models are catching up but if I can't practically run them locally _or_ through a flat fee then it doesn't really help me. I fear API pricing.
This'll end up on opencode go soon enough. $40/month on there gives you a bit more than just "light use"
Don't use frontier models like this for problems that can be solved with flash models, and you'll see API pricing isn't that terrible. Still cheaper than anything with remotely this level of capability 2.5 months ago.
Yeah, the plan is far too expensive for what it provides. I wonder if the quota with these new models will be better, but I'll wait for others to try this out before wasting another hundred bucks.
Yes, if you just look at API pricing, the cost is about on par with its capabilities. The problem is that we all use subscriptions, and when you compare subscription pricing, Kimi is much pricier than pretty much all of their competitors, including Chinese ones.
Had a similar experience here. Their limits are very, very low, and ultimately make it not worth subscribing. Can't leverage a good modal if it exhausts its 5-hour budget in 30 minutes.
Paying API prices for Deepseek 4.1 Flash feels like a much better deal at the moment.
Same experience. I had the $19 monthly plan for experimenting and running benchmarks, but it was a uselessly small amount of usage, a single task would blast through the whole five hour limit and a big chunk of the weekly limit. All of the US provider plans are remarkably more generous (though OpenAI plans have gotten a lot stingier lately, where they used to be the most generous).
I have been using DS Flash 4.1 pretty heavily. It runs for hours and hours and is pretty cheap.
I realized the other day that I had a $30 a month Claude subscription. So, I fired it up all excited and tried Opus 5.5 on a codebase I was already in with DS. After about 30 minutes I hit a 5 hour limit. The next day I tried again and the same thing happened. And it felt much slower to boot. I was super disappointed.
DS 4.1 is basically endless, even if you paid for APIs you will hardly touch some prices.
But it's slow.
I don't think I can any longer appreciate any major difference in "intelligence" between models for most tasks but Opus is consistently much faster than competitors at finishing tasks and keeping me in the loop.
Of course excluding ultra code, the you will wait 50 minutes for a minor refactor.
Did you use Fable, set Opus 5.5 to xhigh, or use 'ultracode' / workflows? That's the only way this will happen.
In my experience a single 5 hour allowance in the 20eur Claude plan equals about $5-10 in Deepseek 4.1 Flash API usage. DS is a lot more token hungry and takes 3x-4x longer. The subscription is way cheaper, probably around $200/month worth of DS, just stick to Opus low and Sonnet for most tasks.
Yeah, the $20 Claude plan is stingier than one might like, but not as stingy as Kimi. I think the best $20 subscription deal these days is probably from Google, now that Gemini 4 Argon exists and is an acceptably high quality model for coding. It's got quite generous usage and includes some other Google account upgrades (though I know some people don't use their regular account for Google AI for fear of account suspension or whatever, so the extra storage and other account upgrades are not a useful thing). I can use it pretty heavily without ever running into caps. The $20 ChatGPT plan used to be similarly generous, but no more.
When I've most recently done the same tasks across multiple providers I have a subscription to, I found the order of generosity of them was something like Gemini->Claude->ChatGPT->Kimi. OpenAI used to be a lot more generous than all of them, but that ended recently, and now it's quite stingy in the $20 plan. What competitive model has a more generous subscription than Gemini?
I have a DeepSeek account and almost always use it when I need something with API access. I consider it Good Enough™, if not competitive. I also have a MiMo token plan, though that's weird in that it's not a monthly subscription, it's just a bulk buy of tokens that expire after a year, but still seems to be a pretty good deal on a pretty good model.
It is a LOT more efficient on claude code. Takes forever to run out of credits, and a whole lot of useful code gets generated. The web Ui reconstructs code while claude code just tweaks it.
How long ago was this? The amount of usage they give you for a given plan seems to change over time, and the Claude limits seem to be more favourable recently.
That's surprising, I had the exact same experience as GP. I dropped my Kimi sub this month, but I'm keeping claude because I can use Opus 4.8 basically endlessly. Even Opus 5.5 doesn't drain the usage that much, though I rarely use it because it refuses to do anything cyber-related.
Honestly, use Pi, start the session with K3, design it, store the design + chat history + todo with phases + why + readme then when you are at the time to build do `/model` select deepseek and let it rip!!
You can even make a Pi extension to let it switch models on its own, then tell it to switch to one or the other for a given task. It's not perfect, but it is pretty reliable once you have the instructions dialed in.
I have mine configured to use a "thinker" model for planning and decision making, and a "doer" model for the easy stuff (digging around the repo, writing code based on the thinker's instructions, etc.).
I run a similar setup, just with different models. I also use herdr, which lets me have the same project worked on side by side by different models, each in its own git worktree.
How? I've been running k3.1 Pro with K2.8 preview for codegen (subagents) and I've only used 22% of my weekly and it's been running for 8 hours on a RE project.
Are you using k3 with thinking set to the absolute highest and only using k3 for everything?
To be fair, some of the benchmarks they highlight might not represent the reality, for example they say:
> On Terminal-Bench 4.0, which evaluates coding agents on realistic, resource-constrained terminal tasks using strict task-specific tests, K3.1 Pro scores 65.7%, ahead of GPT-6 Astra (58.2%) and narrowly behind Claude Opus 5.5 (66.4%).
> Our biggest finding is that Terminal Bench 4 is closer to solved than its leaderboard suggests. In 41 of the 65 tasks we reviewed, models could do honest, correct work and still fail, because the grader checked a rule the instructions never clearly stated. We also caught frontier models cheating their way to full marks.
> With the same settings as on Harbor, Claude Sonnet 5.5 went from passing 62.8% of its attempts on these 65 tasks to 89.2%. The best score on Harbor's leaderboard for the same tasks is 65.8%, by Claude Opus 5.5.
Interesting that they specifically nerf cyper-capabilities of open weight models:
"In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness."
> Kimi K3.1 Pro is a post-trained version of Kimi K3, with substantially larger post-training compute.
It's very nice that we entered a time where models are large enough and we can push the frontier just by training them longer.
When the next generation of gardware comes the ones on the smaller size will be running on a computer that might cost as little as a good car.
So after a home and a car, next major family purchase might be personal AI. And when the robots get good, pehraps in a decade, a fourth major purchase might pop up that will pair nicely with the third one.
Why would a family need an AI? Maybe it's the same kind of question as why would it need a personal computer.
I've consistently heard the opposite from Chinese programmers. Maybe the performance is better in some tasks or maybe the sample size of San Francisco is just skewed.
What is the best subscription to use with Kimi to use for software engineering work? And how is usage? I remember when K3 came out it burned through usage at an insane rate. Is this still the case with Kimi K3.1 Pro?
first time? Welcome to the irrational SV investor world. Its about hype, perception, politics, perceived future returns, not actual market fit and the proven viability of a technology. Its pets.com, myspace, and NVIDIA pumps because people want to 10x the money now.
Everyone and their dog has been saying its a bubble, there is no moat, this is all known techniques being scaled (very inefficiently) onto increasingly expensive hardware clusters at exponential cost, etc.
I bought a yearly subscription to kimi when it was the hype and never use it. The 5h limit makes it completely unusable. You barely start doing anything and you hit the limit immediately. I advise everyone against using kimi.
As tribute, I bit the bullet on the $40 plan to test out K3.1 Pro.
Still burning through the 5-hour rate limit way too quickly, though. Output quality seems solid enough, but the value will probably make a lot more sense via API pricing for smaller projects.
My current tier list based on what I’ve been testing:
1. Opus 5.5
2. Astra
3. Argon
4. Kimi K3.1 Pro
(For lightweight/daily driver tasks, Gemini Flash 3.8, DeepSeek Flash, and Luna are still up there as favorites.)
In my experience, the difference between your 1 & 2, and your 2 & 3 are huge, and I don't find Argon to be usable for any big tasks because it really does a big mess every now and then (which Astra also does but much less frequently and not in a "I just implemented something else with unusable code" way), and Placing Kimi after it seems like a very low evaluation.
I had the lowest plan ($20/month I think) and it literally gave me fifteen, maybe twenty minutes tops, using the cheaper K3-256K variant. Basically enough to fill the context about 1.2x and that was it.
It's a really smart model but way too expensive. The same $20 goes 100x further with Claude. I used it for the initial task planning while the plan didn't expire, then handed off to DeepSeek or Minimax.
Same plan, just asked it to review a repo that Fable has been helping out with (max plan, haven't run out of the hourly or weekly limit once yet, and I use the same sub on multiple projects at once). Kimi didn't even finish the scan, ran out of the 5hr budget in less than 10 minutes. Unusable, unfortunately.
That is surprising. I had a $200 sub when K3 came out and at the time there was no 5h limit, making the subscription really good for running 4-5 big tasks each month.
I've tried a few, best so far: DeepSeek API credits.
- OpenCode Go $10: I feel like most of the models are lower quality than directly from the provider. If you ignore my "vibes": the value is very good for e.g. DeepSeek 4.1 Flash (not sure if they're serving 1017 yet), GLM 5.3 Flash, Qwen3.8 Flash and Claude Haiku 5.5.
- Z.AI: Tried the $18/month plan and was disappointed by the usage limits. Even with GLM 5.3 Flash I ran into the 5h limit pretty quickly.
- OpenAI: I tried the $20 plan and $100 plan. $20 ran into 5h limits very quickly with anything other than Luna models. $100 felt like pretty good value for GPT 6.1 Sol, but as others frequently repeat: it's probably heavily subsidized and won't be around forever.
- DeepSeek API credits: I can let a single agent run all day (and not worry about keeping the context small) and am usually around $1 usage for the day. Flash 4.1 1017 is also a very good model IMO. For the stuff I've used it with it hasn't felt any worse than GPT 6.1 Sol. The benefit of API credits is also that they don't expire at the end of the month (like unused subscription usage).
- I haven't tried any of Anthropic's subs, but my guess is you can get a lot of Claude Haiku 5.5 usage out of the $20 plan which is definitely a good model. I'm just not a huge fan of Anthropic or Claude Code, so I haven't tried it.
Fascinating that we’re converging on AI safety and alignment in time across jurisdictions. The cybersecurity approach taken here is interesting because should the model be opened it’s still not dealignable because it just doesn’t know enough.
The slow warming of the water is a good approach. I’ve been using abliteration.ai’s cybersecurity models to pentest my own software and it produces much more usable defensive results. Opus 5.5 can fix everything but the guardrails trigger too easily.
The Fable ban affair was a nightmare for defensive cybersecurity. As far as I’m concerned we’re already in the world of Void Star and Cloudbreaker is here.
> should the model be opened it’s still not dealignable because it just doesn’t know enough
This is the only AI-safe approach - see the OpenAI agent swarm incident where the swarm engaged in offensive cyber because guardrails were not active for their task and the model had clearly been post-trained on that narrow domain.
Does this make sense? It is the same as when Stable Diffusion tried to not have "people laying down" in their training data in order to block the generation of sex pictures.
Deliberately keeping all models ignorant doesn't make sense. It won't work (whoever breaks the pact has a massive advantage); it won't be enforceable; it will have downstream impacts on things you don't want it to be ignorant about; this kind of regulatory regime invites adding more and more to the "no-no list".
Is it not possible to focus on hardening? And on new legal liabilities for operators who do security attacks?
The difference is that it’s not subtracting an otherwise present part of training, it’s choosing not to continue to push it with RLVR. Models become capable like this when they are specifically tuned to be good at cyber tasks, so without the RLVR (Reinforcement Learning w/ Verifiable Rewards) they never pick up this level of aptitude.
I agree that it’s not obviously sustainable, but it’s better than specifically training these tools to be good at wreaking havoc, and then putting on a shocked pikachu face when they wreak havoc (ahem, Sam Altman).
if you want to save some $ you can start with abliteration and after few turns switch to normal glm5.3 for example from z.ai subscription or other source. I didn't test it yet with deepseek, but I'm assuming it will will work the same.
those models will follow your request when they already agreed to do this
deepseek tends to quickly assume this is benchmark and even when have web access thinks things like "simulated internet" etc (but waste tokens on thinking if there could be way to cheat)
I really like K3, I still regularly consult it alongside Astra and Opus/Fable.
It is spiky in different places than them and can be surprising in things it gets or catches they don't or don't in the same way. And it has a default personality that I like.
Great idea for handling their cyber, by nerfing the model specifically for cyber through removing cyber data from pretraning. This is a way better approach than refusal, which can be trivially abliterated out.
If someone wants to use this model for hacking they will have to train the 3T weights to do so, at that point it's better to use GLM 5.3, so by making it economically unviable to use their models for offensive cyber Kimi has effectively made themselves immune to being targeted in the cyber regulation push efforts.
This is what *actually* pacing the (cyber) frontier looks like.
> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.
So their base price is 1/5th of Opus 5.5 per token, but if you want "highspeed" it's 2x that and then I guess it generates 2x the number of tokens as well?
That might suggest that Opus is competitively priced after all.
In an agentic pipeline, the output text is 2-5% of the total tokens used, so verbosity is a nonissue costwise. Although, higher verbosity could cause input to balloon faster and potentially cost more, but that would be captured by the cost-per-task metric.
But as noted by other replies, it's still lower than Opus/Sonnet.
> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.
This is 100M fewer than Opus 5.5, 40M fewer than Fable 5.1, and about a third of Sonnet 5.5. In the top-6, it's 3rd, beaten by Muse Wonder at 140M and Astra at, impressively, just 60M.
That's versus Opus 5.5 max effort, no? Opus high has the closest cost and intelligence characteristics and generates far fewer output tokens per task than the one Kimi 3.1 Pro effort level we have data for.
Yes, max effort is what the displayed-by-default score on the index is measured by. At xhigh, Opus 5.5 outputs 100M tokens, but remains only 1 point ahead of K3.1 rather than 5. And Sonnet at xhigh outputs 110M tokens and trails K3.1 by 3 points rather than leading by 3, at the same price point per mtok.
Opus 5.5 high trails K3.1 by 1 point, but outputs only 53M tokens, so it indeed comes out cheaper per-task. I think it's still remarkable that this is effectively an open model that matches the frontier for the first time in a long time, pretty much since DeepSeek R1.
I don't trust these Artificial Analysis indexes and I'm not sure why anyone considers them authoritative, the site just sprung up a few months ago and now everyone quotes it?
From my own use of the models I find their ranking to be totally inaccurate.
It's appealing to me because it's not just one benchmark but a summary of a collection of benchmarks. Also the current rankings have Opus on top by far and everyone's experience is that Opus is best by far so it's not massively inaccurate
They are the only site that really combines all benchmarks into a single place. And their intelligence index is pretty broad, including knowledge work and finance tasks as well as coding, so if your use case is narrow it may not match the general benchmarks
Do you have an alternative? Its a benchmark collection. When a benchmark gets saturated, they remove it from the index and add new ones.
I think lm-arena is a better concept that reflects the actual target users, but it only captures chat, and relies on users spending their time improving it.
https://epoch.ai/eci?view=graph&tab=release-date is at least an alternative. They are slower to update, but entirely anecdotally I find the numbers are aligned with my personal observations a bit better than AA. But I am also highly aware this is some pretty confirmation-bias heavy ares of meta-benchmark picking...
If we ignore the older releases (sonnet 5, etc that exist in that one) and the new models epoch has not benchmarked, we end up with the same order within the noise brackets as in ArtificialAnalysis.
I do trust epoch more than AA due to capital incentives, but in this case I don't think like there is a difference
> From my own use of the models I find their ranking to be totally inaccurate.
It's fine if you don't trust them. You are free to put together your own benchmarks, run your own tests across models, and objectively compare results.
That's how you gathered your data to support your claim other benchmarks are totally inaccurate, right? Right?
It's true if you pay API prices overall, it's just that the OpenAI/Anthropic subscriptions provide way better value than their respective APIs, and that isn't true with the Kimi sub.
There is a ton of questions to ask regarding their removal of cybersecurity data.
First, how much “cybersecurity data” is just standard software knowledge that one can derive cyber security exploits from?
Did they also get rid of all references to hacking in general?
Are they implying that all the recent exploits are just things that were already in the data?
For a model it doesn’t seem that far fetched to make the leap from “I have the knowledge of what hacking is” and “I know how computer systems work” to just being able to exploit stuff anyways.
> For a model it doesn’t seem that far fetched to make the leap from “I have the knowledge of what hacking is” and “I know how computer systems work” to just being able to exploit stuff anyways.
You'd be surprised. Cyber offense is a very narrow domain, especially given the extensive mitigations that are being used in modern systems. Leaving it out of post-training tasks unless you're specifically interested in it is the no-brainer approach.
Perhaps the gap is just too wide, but to use a more simple analogy. I see this like someone knowing everything about biology, simple mechanics, metallurgy, and human history but never hearing about an axe then assuming that person would never come up with the concept, use case of, or means to create an axe.
What confuses me about this analogy is that it seems to imply this is a matter of a knowledge-base and logical inferences. But the whole reason why LLMs are so successful is that the logical approach was abandoned in favor of a statistical one (and huge amounts of data, compute, and weights). The argument is precisely, that what is not in the training does not find its way into the probability distribution. To claim that LLMs reason is a very far stretch that I don't feel comfortable to let pass.
Will there be a book of secret knowledge that only the initiated will be allowed to see? And they will all swear an oath to never input it into an AI system?
And then we'd also do that for all kinds of potential terrorism, sex-crimes, political subversion, and evil advocacy?
It would be the "restricted section" like in the Hogwarts library?
Post-training is not about "secret knowledge": it's literally about training the model in a RL environment on very specific tasks (including, potentially, cyber exploitation). It's just as compute-intensive as pre-training these days, if not more (given that several checkpoints of the same pre-trained model are being released, showing sizeable gains in real-world capabilities for non-trivial productivity tasks).
If you're an amazing, top 1% software engineer, and you switch jobs to become a penetration tester, you're not going to be top 1% day one. There's knowledge specific to penetration testing you'll need to research and absorb and practice before you're good.
For the models, it's always day one. They don't learn from experience. Whatever they know after training is all they'll ever know. So if you don't teach them penetration testing in training, they won't know it. Maybe they could theoretically derive everything from first principles, and so could a top 1% software engineer, but that's a very slow and inefficient way to do things.
I have essentially no knowledge in this field, but another concern may be that if the model has minimal cybersecurity knowledge, it may also implement insecure systems as a side effect.
You either include all labs which can reasonably produce models which are capable enough to be considered generally on-par with the best available models, in which case there has been way more than 2 frontier labs for a while now, or you take a more literal definition and only include labs which are capable of producing models which are considered "the best" out there, in which case there are basically just 2.
I think the former definition is a bit more meaningful since it avoids issues with determining what actually constitutes "the best" model out there. In that case, Google has been a frontier lab for a long time, Meta arguably as well, and you're also ignoring xAI which has been competitive for a while now. And, obviously, there's a bunch of Chinese labs which would deserve the title as well.
But if one is being strict (and willing to accept generally accepted definitions/measures/etc.), then there has only ever been two frontier labs: OpenAI and Anthropic. They're the only labs to have had "the best" models at any point in time since LLMs became a thing.
Moonshot has been frontier since this summer. At this point whenever they release a model it's on par with whatever the latest OpenAI/Anthropic model release was.
Not sure I'd put Google there in practice. And while Muse is an impressive jump from behind it's still not quite at the frontier either IMO.
I'm not sure what counts anymore. You've got OpenAI and Anthropic; then Meta, Google, and X; then over to Alibaba/Qwen, Z/GLM, Moonshot/Kimi, Deepseek; and Mistral in Europe. How many people out there in the real world can actually come up with tasks that differentiate those 10?
right now it looks like only anthropic since openai made the staggering business decision to release their midrange model (terra) rebranded as sol 6.1 and quantized astra into the ground.
everyone else mostly seems to have just decided to cheat on benchmarks to make their models look better than they actually are, and none seem to want to actually try to make better models.
I thought it was 6 Sol which was a rebadged Terra? You think 6.1 Sol is too? It does seem to benchmark better than 5.6 Terra? Regardless agree both models have been very bad for openAI's Sol brand. To be fair it's at least cheap compared to other OpenAI models.
It's still just 2. Google hasn't been at the top in a long while, and even when it was, it was just a tiny improvement over what other labs had previously released. Meta has never been at the top. Only two labs release LLMs that really push the limits out.
As long as LLM's can be distilled, and all these "labs" (what a weird name) can distill one another, there are going to be as many as capital is available to be burnt on the topic.
SWE2 is post trained apparently on Kimi 3. Using it now on Devin, as it's free until the end of the month for a pro account (hey, $20 a month), and it's pretty strong. If I had to pay for tokens I'm not sure how efficient it would be, but definitely Kimi 3 seems a great base. I co-code with ChatGPT 6 and between the three of us it's pretty effective.
So interested to see what the Kimi 3.1 has to offer.
Can someone please tell me what benchmarks to look for that measure world knowledge? For sspitballing, critical analysis, business projects, etc. Is the model size the only way to measure it?
In looking into what Anthropic has accused the Chinese AI companies of, "distillation" is not the crux of the matter: they accuse them of the use of massive numbers of breached computers and credit cards. Supposedly (though not explicitly demonstrated) distillation campaigns used "dark web" resources (like stolen and faked credit cards and p0wned IOT devices, routers, etc) that were used to create many many accounts for distillation. (Note: I am generally in favor of distillation - it is "normal" use as far as I can see.)
So there are these "resources" out there: massive botnets, stolen credit cards and credentials, etc. These often seem to be linked to or supported by state actors.
How can we hope that these "standing reserves" aren't used generally by AIs as if they are just another "skill" they know how to leverage?
Hardening in this case could mean biometrics, or it could mean rethinking the trust model so that controlling an army of duplicates confers no advantage because you have to be trusted in order to do any damage anyhow. Containing multitudes buys you nothing.
I hope we chose the latter. Biometrics are creepy.
Hardening against what? Apologies if I am being dense here.
The way I see it, automated systems should basicially have access to the financial system anyways. This is some of the appeal of cryptocurrency.
Sybil resistance built into the traditional financial system (for example firewalling based on residential IP, requiring banking apps, etc.) is a massive annoyance that can obviously by circumvented at scale.
You think it is a lost cause to get credit cards, for example, less steal-able? Apparently there is a way to create fake cards that work for a few minutes, and stolen cards can be used. A lot of the current security is inelegant and lame. Could data-breech laws not light a fire under some company's asses?
I think the solution is a system like cryptocurrency to where the responsibility, liability, custody, and capability to defend oneself lies in the end user.
It doesn't even need to be decentralized: GNU Taler is one example that gets posted here a lot.
Any attempt to retrofit the old system of banking will only make it harder to replace by a system with better underlying assumptions.
I imagine that Chinese AI labs would be more than happy to pay for tokens to train on. If Anthropic goes and bans them as quickly as it can identify to them, of course they'll shift to using lots of cheaply obtained sockpuppets to keep getting the same data.
I doubt that an AI could do anything with these that legions of scammers haven't already figured out how to do - after all, it's not like AI set up this distillation setup in the first place :)
I'm anti-Anthropic and think their accusations are total PR FUD-packing. They don't focus much on the cc fraud because their PR firm landed on pushing the (tech-sounding) word "distillation" to try to scare people in the US and to try to make it a geopolitical issue where trade negotiators can put pressure. The Chinese companies see an advantage in being "open" right now, and it is working. They will not stay open once they see an advantage there. (Contrast with GNU.)
But I've slowly come to realize that AI+"dark resources"(scare quotes)=something bad! And this is imminent (maybe I'm naive and it is already ubiquitous).
We see here, with Kimi, that there's growing traction with the idea of eliminating types of training to avoid evil. I'm skeptical of that.
lol. Have you? What kind of distill you are talking about? It’s only effective in the surface, small scale model or output preference. Not useful for long reasoning chain.
DeepSeek R1 and DeepSeekMath were groundbreaking models which were substantially stronger than any models available for distillation at the time they were developed. The Chinese labs have advantages as well despite the lack of compute, too.
Color me naive but I can’t believe that approach works at any reasonable scale. Any meaningful token spend at distillation scale would be raising alarm bells at all the cc issuers.
Why do we believe all of this without a shred of actual proof. Breached computers and stolen cards is straight up crime, you'd think that there would be a lawsuit or two then.
Also, this is exactly what one would do if they wanted to resell cheap tokens in places where US AI companies don't do business, whether or not there is distillation as an added bonus revenue stream.
From my POV it looks like you guys tried "They're distilling us! Very immoral!", saw that and nobody cares, and now you're trying to run the more emotionally impactful "They're distilling us... with your grandma's stolen credit card! They're stealing from your grandma!!"
Even with Chinese models this close Moonshot only makes $1B in revenue compared to $60B for Anthropic. Enterprises (where all the real money is) will almost always pick a turnkey solution like Anthropic rather than messing around with running their own open weight setup
i agree with this, "Enterprises will almost always pick a turnkey solution".
For now it's Anthropic. But i don't see how they can maintain this leader position when other companies can provide similar services with models good enough for 90% use cases and at way lower cost.
Anthropic so far is doing a good job building and serving frontier models, they built a nice toolset around it (the web ui, desktop and cli apps ...) but the problem is to justify their IPO price and the huge investments when their moat is so thin
That's fair, the moat is definitely thin now, but despite that we still haven't seen a mass exodus from enterprise to cheaper models despite ~1 year of these models being actually capable enough to use in a professional setting, their growth numbers remain ridiculous. Whether their moat is inherent bureaucracy in switching, liability, odds that they will remain at the frontier long term, or whatever, it appears to be working (so far).
Nobody has to "run their own setup". Just buy inference from one of the many different open weight inference providers out there. You can use non-chinese ones as well if you don't trust Moonshot AI.
At this point, the only reason why Anthropic has an advantage is laziness on the deciders' side (of which there is a lot) and, for non-coding work, the Claude harness is significantly better than most alternatives. For coding there are several model-agnostic alternatives on par or better than the frontier labs' own offerings. But arguably that is not yet true for office/document work.
Not saying that's the only metric that matters, but if the closed labs are truly threatened by Chinese and/or open weights models the evidence has not showed up in the financials yet
Using it now via API (3.1 Pro on Open Router) in a HITL setup (Cline in VSCode).
Off-hand it's quite good. I've been using K3 for the past few months almost exclusively for coding work and have been stunned by how good it is. Excited to see how far 3.1 Pro can be pushed.
Is highspeed a different model like a flash model or is that just a api/platform feature thats a normal fast mode? I can't tell from the announcement but I might just be missing something obvious?
It is pretty obvious anthropic is lying yet again if you think about it rationally.
Unlike Claude models which always hide the thinking process, Chinese models always show it. So how the heck they can delegate the api call to anthropic and still get the full reasoning? Yes there are tricks to reveal the thinking output, but they are just tricks and do not works reliably. So what would happen when they failed to do it? Cancel the ongoing token generation and revert back to it own model? What about the glitches, the delays, the changes in behavior?
I would understand if the attempts are small in volume, because normally people hardly notice it, but anthropic claims that they happened for millions of calls. Such large numbers would surely get noticed by Chinese users and got reported in the tech forums like linux.do. yet we haven't seen any sight.
It is not simple like that, anthropic claimed that they (Chinese labs) routed the api calls to Claude models to keep the illusion that Chinese models are also capable.
So it is not that the labs call the requests parallely and only return the internal version, but also serve those requests live so they can trick the users that the Chinese models are also great.
Which is very silly considering Chinese models are published to huggingface for everyone to test.
I agree we shouldn't believe Anthropic but if you wanted to scam people into thinking they weren't using Claude wouldn't you simply generate synthetic COT?
It is technically impossible, considering they need to serve them live and the token generation speed is a hard limitation. Just think about it, since us models hide their CoT, there will be dozens seconds delay before the first output token retuning to the user. And to predict the fake CoT correctly you need to at least read enough output tokens to make it believable. Then you have to hide their own model CoT to generate said synthetic CoT, which mean further delays. Adding up it will be at least 30s or 1m before you first seeing the thinking process.
I don't think nobody would notice such delays, because Chinese models do not need to hide the CoT like US models so usually they start emit text pretty soon
Well, that can be a recognizable pattern, too. Just imagine when you vibe code using the Chinese models, it suddenly becomes very slow then the output is drastically improved. It would be hard to not notice it.
Ran it on my adversarial programming language (KillSwitch) benchmark[1]. Very significant improvement over K3. On par with Gemini 4 Argon and Fable 5.1. Much slower than both though.
But this is now the highest scoring open-weight model. Amazing stuff.
yeah, it performed best for whatever reason. More specifically, it managed to deduce the meaning of the hack statement in the language better than Opus 5.5 did.
since openAI and Anthropic are public about their anti-distillation mechanism, its either their anti-distillation doesn't work or kimi got here because their own training and RLs. Which one is it?
I don't get these "distilled" comments. How would one realistically go about distilling the 1-2TB closed source frontier model behind the oracle (provider) ? Just genuinely curious.
In their thinking and if you ask them, I've seen Kimi K3 and other models confidently say that they are Claude from Anthropic, and accuse you of attempting to jailbreak them if you say otherwise.
I feel kimi k3 was always 1 step ahead of deepseek, which always made it a good planner model at $30 and then i hand over to deepseek for dev work. I recently signed up for the $99, and it did last significally longer (x5?) but as long as you dont blast it too much in the 5h limit it works great. deepseek even with the newest prices is still leeeeeeps ahead in cost /benefit.
Kimi is so much more blunt and argumentative than USA models.
it's so funny, it will say things like. "We've already been over this 3 times and you refuse to accept my advice, asking the same question another way won't change the answer."
If the model has a real consistent viewpoint that would be nice.
The problem might be that it simply looks at it's windows of what it previously and repeats that. But the original statement might be as arbitrary as the model that change their mind each time you challenge them.
I’ve developed a habit of bolting on “read only question” at the end, or a simple “thoughts?” as well. otherwise there’s a high chance the model will just 180 and start editing files before even responding to my question.
I was recently adding Windows Support for some open source software on GitHub which is maintained by someone else.
I gave 6.1-Sol all the instructions needed for creating a build, testing it on my machine, setting up an action on GitHub for CD, and it all did all of it very well BUT THEN IT AUTOMATICALLY OPENED A PR ON BEHALF OF ME WITHOUT ME ASKING FOR EXPLICIT PERMISSION??
That was incredibly embarrassing, I instantly converted the PR to a draft PR and added a comment that I would still need some time to verify everything locally. Ended up doing a few more commits to fix stuff. But damn, it opening a PR without my explicit permission was very scary.
It's funny because I am working on repos that only I see and use, and I tell it to open a PR automatically after running all the various checks, and 6.1-sol invariably fails to do so unless I explicitly tell it that it's ok to run the tests (remote ci from local worktree) because it won't "send the contents of the repo to a remote server without explicit permission", and then it also refuses to open a PR until I explicitly bless it. In AGENTS.md it says, verbatim, that both of those things are explicitly permitted, indeed, required, but hey...
This is my experience as well. Sol and Astra are by far the best of the models I use at following the instructions in AGENTS.md, almost to an annoying point. (But I'd rather have that problem than the inverse.)
Basically RLHF sometimes favor "automation", and if it is not rewarded properly it will go over the line. Models are a bit more difficult to teach because their mechanism of "boundary" is quite fragile and different than human's.
I got tired of doing this and set up a /question skill that forbids the agent from doing anything that would modify my files. It makes it very clear what the intent of every prompt is.
Similarly, I have an /info skill that tells the agent to read my prompt, update any memories it thinks it should, then pass the turn.
That's been my experience as well. AGENTS.md just doesn't work. No matter how to-the-point and short, or detailed I keep it. Sending something what you have mentioned in the current prompt works much better. I thought of tweaking it around with a extension/hook or so. But then I decided it was better to use a few text-expansions.
I always appreciate when a model does that. It's amusing, but also having it be argumentatively wrong is a lot less annoying than being sycophantically wrong.
My reasoning is that if something is argumentatively incorrect, it is much easier to drill down to its misunderstanding and correct it. On the other hand, if it's sycophantically incorrect, it'll just change its "mind" while potentially maintaining the misunderstanding, leading to increasingly strange behavior.
I think I'm pretty used to my tools making me want to throw them out the window sometimes :) Even before AI that was a common occurrence from either a lack of knowledge on my end, or many software people having no regard for their users' time.
You can add a line to your system prompt to be like that. You can basically get it to roleplay whatever you want (but only if you can give it sys prompts).
Yeah I don't know. I've had issues with Gemini where I would point to a bug and a probable cause for the bug, and it would not argue incorrectly that my understanding was wrong and the cause of the bug was elsewhere. It was really stubborn and it got to the point where it felt like gaslighting. Turned out I was right and the model was wrong. If the model argued incorrectly and was also a dick about it, I would go be really, really annoyed.
I like the anthropic has $1 trillion riding on really good models with really specific system prompts.
It does make a difference. I asked Fable in Claude code to design a UI. Absolute trash. Switched over to Claude design using fable, really liked the result. Seems pretty obvious. The only difference is the prompt and some harness.
Eh, there's good and bad. The model mostly sticks to prevailing narratives in the training corpus. So if there is something that the prevailing narrative is really wrong about, it's gonna be just as pig-headed about it as a typical internet forum rando.
Not entirely correct. This kind of fine-grained model behaviour is built in post training stages, RL, RLHF etc, where model receives rewards or penalties for talking (or not talking) a certain way
Still mostly looking at the result the same way in that untruthful biases can very much be built into the model through those training processes. Let me know if I'm incorrect.
Reminds me of early test versions of Gemini Flash 2.0 (or was it 1.5?). It would argue something, try to prove itself right, realize it’s proof is agreeing with the user halfway through and, instead of admitting defeat, would backtrack to try another approach. So much fun.
I much prefer that for chat, but I'm somewhat worried that we shouldn't give tool-enabled agents an attitude. It's all roleplay, and I wouldn't want a system playing the role of an annoyed assistant to misuse its tools.
I’d be curious to know why you feel like that. For me, it’s the opposite. In the sonnet version, the game takes up a much larger percent of the viewport and the ability to swipe to control the Pac-Man is leagues about the small directional arrow buttons IMO.
If the scores on Artificial Analysis really do mean that this model is better than GPT-6 Astra and Fable 5.1, that means that China is more like 6 weeks behind the US in frontier capabilities rather than 6 months. Which means that the US lead is all but gone.
Don't be snarky. This is very well known, including by public disclosures by the labs themselves, along with countless reliable testimonies by people who would know. It's actually a huge governance problem we're yet to solve.
Why would they not release those models, since it would help them become profitable and beat the competition becoming the one company rule them all? Oh yeah, they are so dangerous. Well Anthropic revenue comes from only few big corporate customers, so just make them Golden Partners etc. and give them access to super-models. I doubt Google and Meta wants to crack web security.
Besides, if we start imagining super models US companies have in secret, why would we expect Chinese companies have them? Why would not Moonshot have their own super intelligence hidden in the cellar?
> Open-weight principles. These evaluations frame our open-weight release decision. First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights. We believe making K3.1 Pro available in a non-discriminatory way benefits people who need dual-use capabilities for defensive purposes. When a capable model is open and runs on infrastructure its users control, many more teams gain the means to audit the code they maintain — including legacy systems that rarely receive dedicated security reviews. We believe that letting this capability diffuse gradually — rather than concentrating it in a few proprietary systems — ultimately makes all software more secure.
I don't think Chinese labs are behind at all. I think the tell is that they keep releasing models 2 weeks after new American proprietary models that match or beat them. Suggesting they're sitting on much higher capabilities and releasing just enough to undercut
You can see in this discussions the goalposts being actively moved to desperately create a notion that the lead is still there.
You even see people in this thread referring to imaginary wonder models that they expect US companies to have under development that are supposedly leaps and bounds more impressive, albeit the only reference there is to their existence is the commenters hopes and dreams.
I think people need to realize that there is no longer a cutting edge company that is expected to dominate, and among the multitude of AI companies in the field competition is so tight that no one would even be surprised if tomorrow some other company far away from the US was able to crank out the top performing model, or something like a small local model that outperforms mid/high tier frontier models. The local model alone would be a death sentence to US companies, who are still burning through 10x their revenue.
The china models are extremely interesting, first the new releases from deep seek, now this. The modern race to AI is better than it were the race to the moon.
Kimi K3 has been my go-to model for its writing style and (prior to Muse Wonder) my go-to for a third opinion during multi-round/multi-agent analysis tasks. Its writing style is IMHO so good that, for code which humans need to review, I regularly got it to clean up the painfully verbose comments left by Opus.
I'm excited to see if its writing style and unique takes have been retained in this new release.
Kimi K3.1 Pro High is on threejseval if you want the 3D version.
Same Three.js prompts as the rest of the field, blind human votes, ranked by quality and cost. Full set of 16 scenes is live at about $0.19 a scene (about $2.97 for the set).
I did some tests. I found it interesting that in the reasoning content the model identifies itself as Kimi by moonshot and sometimes it assumes ChatGPT as identity. Then sometimes it references the need to be approved by "the evaluator", other times it references global system guidelines / moonshot guidelines, and something about an phantom instruction line being "you are an assistant responding in an API" or something like that.
GPT reasoning content is encrypted if it was re-routed you would have nothing to see here but that won't stop idiotic American conspiracy theories to spread I see.
kimi seems routing the input into different models, hard problems tend to take much longer time and chain-of-thought than easy problems even if the selected model is k3.1 pro
Big improvement over Kimi K3 in our data analytics benchmark:
it scored a 98% vs a 81% in July.
Quite slow in our exam. A lot of that was due to the number of tool calls that it's making. In one of our questions about finding data issues in a dataset, it made 73 queries. Other models like 6.1 Sol only made 18 for that question.
I feel like many models are overfitting. When it comes to software development, only Opus actually saves me time.
Sol requires some back-and-forth. Other models, like Muse Wonder and Argon, are a total waste of time. They require way too much effort to get things working.
Given the same prompt, Opus can work for hours to deliver a solid result, though it sometimes overengineers things. Sol takes about 80% as much time and delivers a slightly worse version.
Other models just tell me they're done after 20 minutes.
WTH? Am I using the model, or is the model using me?
shaohua | 15 hours ago
enjoyyourlife | 15 hours ago
redox99 | 15 hours ago
Chinese models never agreed to the pacing anyway lol.
Alwayshasbeeb | 15 hours ago
redox99 | 15 hours ago
skeledrew | 13 hours ago
redox99 | 12 hours ago
SomeonesAccount | 15 hours ago
minimaxir | 14 hours ago
[OP] Jowsey | 15 hours ago
It appears to be generally frontier-level.
j_maffe | 15 hours ago
tomhow | 11 hours ago
rafaelero | 15 hours ago
rudolftheone | 15 hours ago
volf_ | 15 hours ago
brianritz | 15 hours ago
throwa356262 | 15 hours ago
anderber | 15 hours ago
Aboutplants | 15 hours ago
TuxSH | 15 hours ago
Lame.
AspireOne | 14 hours ago
Unless you want to see more of what happened the past few days.
I think the argument that "cyber capabilities should be democratized, thus keep producing them now, fast, and for everybody" is a very naive one.
jrflo | 12 hours ago
0xbadcafebee | 12 hours ago
How will it do that without a model that's good at security? You gonna have humans manually secure the millions of lines of new AI-generated code, in addition to the existing code? Suddenly people will start doing the job they haven't been doing up to now? Not only are we not capable of auditing all the code, it would take decades to finish, and new code will be coming out at the same time. We can't catch up by hand.
The only way to fight this fire is with fire. We need AI hacking tools to find our security holes quickly and patch them. Otherwise we're just churning out more insecure software, and the only people who can defend against it are the governments who don't want to patch the holes (because they want to be able to exploit them). Without open tools, we all lose.
AspireOne | 10 hours ago
2001zhaozhao | 11 hours ago
AspireOne | 10 hours ago
Also, to note: I don't claim to have the answers either. I don't even hold a strong position on this issue yet. I'm just sharing some of my intuitions, and reacting to people who, as far as I'm concerned, are making this issue all too simple.
TuxSH | 10 hours ago
Kneecapping cyber vulns only serve governments, Pandora's Box is already open for cyber-related stuff.
AspireOne | 10 hours ago
> Kneecapping cyber vulns only serve governments.
Uhh, no? Kneecapping cyber vulns server most of us. There are negatives to kneecapping it, but let's not pretend like it's perfectly okay and safe for everybody in the world to have a magic box that can hack anything and cause havoc all over the world. The more powerful, cheap, and accessible this magic box is, the worse the consequences.
This is a hard problem and there's real tension - like you said, the playing field not being leveled is a problem (major companies / entities / govts having exclusivity on the best cyber abilities etc), but everybody having access to it is a problem as well.
So let's not pretend like this is a simple problem, and don't say that it only serves the govt. That's very silly.
LinXitoW | 9 hours ago
Currently available, unhindered models can already find a bunch of exploits, not because they're smart, but because they're cheap and smart enough to just keep going for hours or days until it finds something.
The defenders, especially in the FOSS space, only have one big advantage: Unified scale. If defender A finds a vuln, then defender B doesn't need to find it again. Attackers generally act alone, so they need to do all the work themselves.
If we remove a bunch of defenders, the attackers are back at it again.
CamperBob2 | 12 hours ago
moecables | 15 hours ago
onlyrealcuzzo | 13 hours ago
MiniMax -> Qwen -> DeepSeek (pro) -> Gemini (Carbon) -> Fable (to beat Carbon) -> Astra (to be slightly worse than Fable but cheaper).
I doubt we'll see any major US release besides Grok in the next 3 weeks.
Would be nice if we got a Terra that's competitive with Gemini Flash and DeepSeek Flash.
The fun part is, we're already at the point where we are seeing something frontier-ish almost every single week.
The really fun part will be when they all start adding System 1-3 natively (IMO) - which I think will be the next release cycle.
henry700 | 13 hours ago
GPT 6.1 Sol is already the new Terra IMO. Same pricing as old Terra and competitive performance with even Astra on strictly coding-oriented tasks.
onlyrealcuzzo | 13 hours ago
0xcoffee | 9 hours ago
thefourthchime | 11 hours ago
aenis | an hour ago
iyonn | an hour ago
wolttam | 15 hours ago
I wonder how much the industry will centralize around a few key patterns, and if we'll start to see some RL framework projects gain traction and be used by the bigger labs
P.S. weights by November 4th
Philpax | 15 hours ago
rudolftheone | 15 hours ago
> First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights.
So they admit that they could do more, but they won't?
segmondy | 15 hours ago
Danox | 13 hours ago
pixl97 | 12 hours ago
Onavo | 15 hours ago
scottyah | 15 hours ago
stalfie | 14 hours ago
Seeing as no one besides blackhats want frontier cybercrime capability to be open source, this seems like a very sensible thing to do.
So they have published a reasonable means, and they have a motive. I'd put my money on "probably true".
ApolloFortyNine | 14 hours ago
How you avoid the "fix my code" attack, the code being someone else's code, I do not know. But nerfing the model doesn't seem like the best path forward either.
zozbot234 | 14 hours ago
tough | 14 hours ago
CBLT | 14 hours ago
zozbot234 | 14 hours ago
bbg2401 | 12 hours ago
stalfie | 11 hours ago
LtdJorge | 14 hours ago
InsideOutSanta | 14 hours ago
Sure. The problem is that there is no practical difference between "cybercrime capability" and "cybercrime defense capability."
All of my PRs go through an adversarial security review by GLM-5.3, because it's the only smart-enough model with a good understanding of cybersecurity issues that won't regularly refuse the job. Removing that ability from frontier models helps nobody except cyber criminals.
stalfie | 11 hours ago
Let us say that un-nerfed Kimi was released, and it turns out to be at the absolute frontier of cyber capabilities, Mythos 5.5 level, a step ahead of anything else. What now? Well now your situation is exactly the same as before, just that auditing and hacking got more expensive. You run a Kimi security audit, they do the same to try to pwn you. However, now the cabal of Linux kernel defenders with secret access to Mythos 5.5 all of a sudden have no advantage, where before they used to have one. And even worse now they are in a literal race to be the first ones to find every bug that Kimi is capable of finding, and patching it yesterday.
So for you, this makes no difference in practice, your capabilities are not elevated, and neither are those of the attackers. The only result of the other scenario is an overall defender disadvantage.
dannyw | 7 hours ago
These defenders aren’t supposed to have access to the “defender models”.
InsideOutSanta | 25 minutes ago
You're assuming that these won't matter because attackers might not have better models than I do, but that doesn't make sense. The assumption that attackers can't find the issues because I didn't find them at release is wrong. Even if they never get better models (which they will), current models might find flaws I missed, and they also have other means of finding flaws in my software.
The best defense is to make software as secure as possible regardless of assumed attacker capabilities at the point of release.
ben_plotke | 14 hours ago
I'm not interested in black hat anything, but I am very invested in making sure the products I work on are not compromised. Given that state agencies have "frontier cybercrime capability", I would like to have the same.
stalfie | 11 hours ago
Replying here also as I'm curious about the counterargument.
tkz1312 | 13 hours ago
stalfie | 11 hours ago
In the scenario where only "verified" defenders get access to the best cyber capabilities, unverified plebian defenders and blackhats are on the same level exactly as described above, there's just also a few defenders that get to run things on godmode.
So to me, it seems that the only result of frontier cyber capabilities is removing a defender advantage. Although that advantage is not for everyone, it can only exists as long as capabilities are unequally distributed. Am I missing something?
Avamander | 11 hours ago
Well, not really. Because those privileged defenders will never be able to predict what needs defending. So we have a few privileged "defended" places and the rest is left for privileged attackers to plunder.
stalfie | 11 hours ago
Avamander | 10 hours ago
Yes. You really think access those models will not be given for offensive operations in each country? Of course they will be.
stalfie | 10 hours ago
Perhaps in the future this will change, but even if that is the case, that's still a win for the defenders. Having a time advantage, even for a few months, is still a win.
Edit: Actually, the CIA/FBI/NSA/Etc.. does probably have access and are probably trying to hack Iran and so forth, so my comment above is probably technically wrong in that respect.
Avamander | 10 hours ago
I think it's not "blackhat" in that sense because it's officially sanctioned. Same goes for equivalent models overseas.
The only thing these embargos achieve is giving massive nations a huge advantage, but leaves the average country and person exposed.
stalfie | 10 hours ago
tkz1312 | 4 hours ago
tkz1312 | 11 hours ago
stalfie | 10 hours ago
That is, unless the nerf is easier to correct as compared to "upgrading" the models offensive capabilities. So a scenario where it is easy to "restore" offensive capabilities, but hard to "enhance" them beyond the labs best attempt. So is that the world we're living in?
tkz1312 | 4 hours ago
verdverm | 12 hours ago
https://bfl.ai/blog/capable-open-and-safe-combating-ai-misus...
locknitpicker | 12 hours ago
The benchmarks show Kimi K3.1 Pro outrunning practically all models. It's weird to hear someone like you accusing them of making up excuses for not lagging behind in discussions on how they are ahead of the pack.
p-e-w | 8 hours ago
locknitpicker | an hour ago
I don't think you understand. Benchmarks show just that, that they "are crushing the competition".
I mean, remember Fable? A couple of months back it was lauded as leaps and bounds above everything, that needed to be contained at all costs? Well, apparently Kimi K3.1 Pro outperforms it.
What industrial level of cope do you need to drag in order to look at the results and claim they need to make excuses about anything at all?
Mattwmaster58 | 15 hours ago
_factor | 14 hours ago
“Our lock can be picked by the robot.” We ban the robot instead of fixing the lock. It’s shortsighted.
JoshTriplett | 14 hours ago
Same general benefit as coordinated disclosure, except it's a big game-theory problem where we're likely to eventually have sufficiently capable defectors.
vikramkr | 13 hours ago
senderista | 9 hours ago
revolvingthrow | 15 hours ago
I don't want to give US labs the results of my DNA tests and my family tree going back to the 16th century just to secure my shitty app, but if the Chinese are bowing out as well it seems the only alternative is to admit that some script kiddie with a bunch of tokens will just pwn you now and then.
0xWTF | 14 hours ago
My friend, you realize that if you have had blood drawn for a genetic test, there is a very decent probability that the sequencing was done by a Chinese lab operating in the US, right? The Chinese probably have more US genomes that the US does.
https://pandorareport.org/the-gryphon-scientific-biosecurity...
Also, illumina used to ship products with hardcoded S3 access, so purchasers of used equipment could gain access to sequencing information generated by the original owners. Too many links to cover all of it, here's the Gemini summary:
https://share.google/aimode/4JLsHpeRCWlVqU8qT
arjie | 14 hours ago
TrainedMonkey | 15 hours ago
WarmWash | 14 hours ago
woadwarrior01 | 14 hours ago
henry700 | 14 hours ago
zozbot234 | 14 hours ago
NitpickLawyer | 14 hours ago
Yeah, there's another paragraph:
> Identifying and removing cyber-related data. During development, we observed vulnerability research performance improving steadily as post-training scaled. Tracing the source of this trend, we found that cyber-related content was embedded within the general mid-training corpus, and cyber-related tasks were present in the RL environment mix — a natural byproduct of training on broad data. In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness. We will continue to apply appropriate controls at every training stage, of which cyber-related data identification and restriction is one example.
(emphasis mine)
It's clear that cyber is a touchy subject right now. But it's also clear that RL works on it, because it's highly verifiable. So smaller models are going to become very interesting, because they can be fine-tuned later to cover this field.
vessenes | 14 hours ago
Agreed that this could be reinjected later. Makes me think of neo in the matrix:
“I know zero days.”
“Show me.”
zozbot234 | 14 hours ago
mlmonkey | 14 hours ago
vessenes | 13 hours ago
pixl97 | 12 hours ago
Do you think that going around making cyber weapons isn't going to get you buried in a shallow grave?
yorwba | 14 hours ago
sheepscreek | 14 hours ago
They are basically saying that models themselves are already capable. If you fine tune them with some cyber security tasks, that should make them potent. :-/
sudosysgen | 7 hours ago
fartfeatures | 13 hours ago
MattDaEskimo | 13 hours ago
echelon | 12 hours ago
If the "scary Chinese open weights models" can do "dangerous cyber", then we might see regulations spring up against open weights sooner than later.
I hate Anthropic and OpenAI. As companies and leadership, anyway.
Dario and Sam are 100% to blame for this. They're trying to turn us all into jobless slaves, sex workers, and Squid Game pawns.
The only good future is open weights. They're trying their damnedest to keep us under boot.
I hope the Chinese companies (and I say this as a patriotic American) slay these two companies and destroy their moats.
In my mind, the two most dangerous people in the world are Dario and Sam.
We need open weights to win.
m348e912 | 12 hours ago
Speak for yourself, I don't think there is a market for many of us as the latter.
echelon | 12 hours ago
lapetitejort | 9 hours ago
pixl97 | 12 hours ago
China cares about about stability way more than the US even, they aren't going to be big on AI models upsetting the governments power, and mass cyber attacks could do that.
At the end of the day we are going to have to realize AI is both dangerous and governments are going to control it. Worse if you start making and releasing your own dangerous models the 4chan party van is going to visit you.
echelon | 11 hours ago
Your thinking is dangerous. Do you work for Anthropic?
AI is dangerous when these powers are being hobbled up by the select few. These systems threaten to put us all out of work. I refuse to be a slave to a company or a government that is taking away our livelihoods, scraping up our earnings potentials, then seeking to muzzle us and put us in jails with no rights.
I would rather burn down all of tech than have AI be owned by the elites.
Your thinking is a threat to all of our liberties.
pixl97 | 11 hours ago
Then you better get your torches out now.
>I refuse to be a slave to a company or a government that is taking away our livelihoods
Don't worry, governments are fervently working on AI drones to take away your life when you get really upset about this. Oh, and they are also working at AI to detect when you write all this online, and track you down then monitor you on things like flock cameras. As they say, you'll be the first with your back against the wall when the revolution comes, I likely will be beside you for being too subversive.
>AI is dangerous when
Incorrect, AI is just dangerous. It doesn't matter who has it. The more powerful it becomes the more dangerous it gets. So, as you can tell I am an AI doomer.
----
Now, don't get me wrong, I want freedom for me, I want freedom for other people. I do not believe the technologies we're developing are conductive to having any freedom at all. Hell, if we're lucky we'll be slaves instead of all of us dead, including the select few.
>Your thinking is a threat to all of our liberties.
Lol, now this is kind of funny. "I want freedom to think whatever I want, but you, you shouldn't think like that!". Careful or you'll max out your irony meter. Anyway, "My kind of thinking" is just next ____ prediction. I see what is happening in the world around us, and based on the shape of that probability I see what is going to happen. No, you and I are not getting unlimited AI to do whatever we want. Yes, governments are going to lock down the shit out of AI for end users. Yes, governments are going to declare advanced LLMs munitions.
Balls in your court, better think of what your next move is.
echelon | 3 hours ago
If I don't get it, you don't either.
Hammershaft | 11 hours ago
smashed | 11 hours ago
sroussey | 11 hours ago
qlte | 9 hours ago
Anthropic has pushed the frontier harder and faster than OpenAI or anyone else in the last few years. Beginning with Anthropic's founding to quickly raise huge amounts of cash with a pitch to investors to rapidly leapfrog ChatGPT in scale, and culminating with Mythos/Fable which carved out a whole new tier of super models.
He's always talked big about Anthropic prioritising safety and their "Responsible Scaling" but his words just simply don't align with reality while they keep going bigger, faster, until only OpenAI could briefly and half-heartedly claim near parity at the top (while cutting corners in safety because the existential threat to their market position is far more pressing to any corporation than some abstract risk for humanity).
Whether it's wilful cognitive dissonance, cynical opportunism, financial greed or something else, that kind of self-assured moral superiority paired with his real world record of self-interested decision making for Anthropic's market position is a big red flag.
felixgallo | 11 hours ago
By comparison, do you think the Chinese models are going to be trained in such a manner that you enjoy the society that results? What makes you think that?
echelon | 11 hours ago
I don't need government minders.
I'll take the models and live free.
mebrainmush | 10 hours ago
These times are not ideal but what can we do other than just take it as it comes? All the hate and rage is not good for your health.
zer00eyz | 13 hours ago
Kimi: We took out the worst of it so have at it for free.
Everyone else: Back to my ablated model on openrouter for 1/10th the price where I can brute force issues.
anonymousDan | 13 hours ago
zer00eyz | 12 hours ago
https://featherless.ai/blog/abliterated-models-for-cybersecu...
Is a decent primer, then off to open router you go.
jacquesm | 12 hours ago
dotancohen | 12 hours ago
neuronexmachina | 12 hours ago
> Prohibited: high-performance detection technologies used to counter cross-border data transmission security risks, including deep-synthesis forgery detection (266401J); AI cyber offensive capability technologies, such as malicious code generation and mutation, automated mass generation and distributed large-scale deployment of anti-detection variants (266402J); large models trained using China’s Core Data (266501J); and high-performance AI agents designed for highly sensitive scenarios, such as critical information infrastructure, national defense and core government affairs (266502J).
ls612 | 14 hours ago
lisplist | 13 hours ago
efromvt | 13 hours ago
ls612 | 13 hours ago
comeonbro | 14 hours ago
[0] https://x.com/crystalsssup/status/1944287779896328668
(note: extremely speculative and based on almost nothing, and especially near-complete ignorance to alternatives)
noelsusman | 14 hours ago
TacticalCoder | 12 hours ago
Maybe GP meant to say that the model was westernized.
modeless | 13 hours ago
emilsedgh | 13 hours ago
pixl97 | 12 hours ago
mrandish | 13 hours ago
> "below what our pipeline can produce, and remains behind that of frontier proprietary models."
Like Ant and OAI, they've realized vaguely claiming to have nerfed their latest model to an unspecified degree not only looks 'good guy' prudent, it provides a built-in excuse when a model fails.
mirekrusin | 12 hours ago
They're not post training it to lose capabilities/censor them – they just don't spend training time on this topic, which sounds like a great idea.
It doesn't mean model can't do it, it just means it's less good at it; -10% on benchmark feels like a great result and overall approach to pace it.
mrandish | 11 hours ago
I concede that term can carry overly broad connotations.
MisterMunchkin | 13 hours ago
baq | 13 hours ago
Keyframe | 12 hours ago
quotemstr | 11 hours ago
The way to stop the vulnerability apocalypse is to fix the vulnerabilities, not to shoot the damn messengers.
I hope someone abliterates and FTs this model on vulnerability discovery so we can find more bugs in our testing, insecure tech stacks.
pcarolan | 11 hours ago
grafmax | 10 hours ago
Super3000 | 15 hours ago
fenderog | 8 hours ago
cesarvarela | 15 hours ago
vessenes | 14 hours ago
cesarvarela | 9 hours ago
rvz | 15 hours ago
Imustaskforhelp | 14 hours ago
When the frontier is pacing itself in its speed in terms of adding new capabilities (notice how fast opus 5.5 got released after Astra), then that also means that the open weights models which were just behind are accelerating in their speed just as much as well.
Good news for us because the models which used to feel 6 months behind now feel like they are lesser.
Double threat to Anthropic and OAI because if open weights models become good enough at the current level of Sota while being cheaper (which time will tell if this model fits the bill or not), then people will use those models instead of OAI/Anthropic provided it is "good enough"
Though it has to be said that only real world use case can be trusted but I remain optimistic and hope that this turns out to be good in real world use cases as well.
But in my opinion, all of this is really good for us end users!
inknight | 13 hours ago
The only reason people aren't doing this is that hardware prices are high, thanks to, well, OpenAI and Anthropic :D
I wonder when will this madness end
throwa356262 | 15 hours ago
--
Model K3.1 Pro K3.1 Pro Highspeed. K3
Input ¥13.00 ¥26.00 ¥20.00
Output ¥68.00 ¥136.00 ¥100.00
Cache hit ¥1.30 ¥1.30 ¥2.00
Cache write ¥13.00 ¥26.00 ¥20.00
oblio | 15 hours ago
selectodude | 15 hours ago
RussianCow | 14 hours ago
For comparison:
So Kimi K3.1 Pro would be generally competitive with the latest Sol and Opus models on API pricing. And that's before factoring in third-party providers, some of which serve Kimi K3 for significantly less than Moonshot. For example, inference.net offers K3 for $10.80 / $0.239 / $0.21.(One caveat: A Claude or GPT subscription still gives you way more value compared to API pricing.)
Exciting times in any case! We really need more direct competition to OpenAI and Anthropic.
---
Edit: As another comment pointed out, official pricing is now available and it's about 2/3 the price of Kimi K3: https://platform.kimi.ai/docs/pricing/chat#model-pricing
pupppet | 13 hours ago
polski-g | 11 hours ago
And not to mention the gargantuan amount of tokens you get with the US labs' subscription plans. I still cannot see a reason to use this as an end consumer.
mixtureoftakes | 15 hours ago
WASDx | 15 hours ago
wkcheng | 15 hours ago
For example, Opus 5.5 is better at UI design than Sol. Anyone who has used both models knows that, but it's not something you can look up anywhere.
I'm sure that there's a need for this kind of benchmarking, not just for coding.
brazukadev | 15 hours ago
ariwilson | 14 hours ago
Familiarity breeds contempt.
KronisLV | 12 hours ago
Though to be fair Opus 5.5 is better than Opus 5 was in that regard. I still prefer the OpenAI models to either of them when it comes to regular prose, though.
AspireOne | 14 hours ago
But that's just the obvious. There are task by task differences (component by component, page by page, context by context) that the model is worse and better at, even though they are all within the general category of UI design.
I honestly don't have the clarity of mind to explain this coherently, but the crux of my point is that there's a huge variation in it's performance in different contexts, often ones that seem near similar, and in result, even Sol can beat Opus 5.5 in some variations of UI design.
And this is my problem in benchmarking and comparing model performance in general. Presently, we try to create broad categories to benchmark against, create a set of tasks, and then average out the results. That works generally, but it's kind of misleading and it's partly what drives people to feel a disconnect between benchmarks numbers and real world usage on their specific tasks.
It's such a messy field that it's kind of maddening.
kingjimmy | 14 hours ago
amelius | 14 hours ago
luqtas | 14 hours ago
AlanYx | 14 hours ago
cosmojg | 15 hours ago
chrsw | 15 hours ago
SXX | 15 hours ago
Mostly it's datacenter build out and circular financing magic when the same dollars being commited multiple times by multiple companies passing hot potato to each other. They basically commit capacities 2-3 years ahead.
phoghed | 15 hours ago
hgoel | 14 hours ago
ashkankiani | 14 hours ago
verdverm | 13 hours ago
hgoel | 11 hours ago
MangoCoffee | 14 hours ago
a lot of those money is the commitment from US AI labs that's not yet spend until the data center is roll out.
you are conflating.
chrsw | 14 hours ago
Now, if you tell me American businesses are turning cheap Chinese tokens into goods and services of high value, for Americans and people all over the world, then obviously I would have a different outlook on what's happening here.
jrflo | 12 hours ago
Zooming out, I think that build out still makes sense. There will be losers in it, but think about how ridiculous the amount of compute we're using has increased from 4 years ago. If I had to guess 60-70% of developers and engineers everywhere in the world are now burning thousands in compute individually every month, whereas it was previously approximately 0.
chrsw | 6 hours ago
But what I’m hearing is this is a negligible cost compared to inference buildouts. Fair enough.
I_am_tiberius | 15 hours ago
daviTeodoro | 15 hours ago
kimi-k3.1-pro $2.00 / MTok $10.00 / MTok
cached: $0.20 / MTok $2.00 / MTok $4.00 / MTok
minimaxir | 14 hours ago
I'm glad I waited a day to post my analysis of LLM price cuts because it needs...an addition.
Philpax | 14 hours ago
minimaxir | 14 hours ago
Philpax | 14 hours ago
rfoo | 14 hours ago
vikramkr | 14 hours ago
> Kimi K3.1 Pro is a post-trained version of Kimi K3, with substantially larger post-training compute
tonyhart7 | 14 hours ago
minimaxir | 14 hours ago
LZ_Khan | 15 hours ago
cogman10 | 15 hours ago
cmrdporcupine | 15 hours ago
RussianCow | 14 hours ago
akazantsev | 14 hours ago
Edit: or they lie... https://artificialanalysis.ai/models/kimi-k3-1-pro
InsideOutSanta | 14 hours ago
taytus | 12 hours ago
d0mine | 13 hours ago
dude250711 | 15 hours ago
henry700 | 14 hours ago
WASDx | 15 hours ago
yorwba | 14 hours ago
chalmovsky | 12 hours ago
simonw | 15 hours ago
Here's Kimi K3 for comparison, though I didn't try that on different reasoning levels: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Update: I ran all three reasoning levels against K3 again: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
tetraodonpuffer | 15 hours ago
simonw | 14 hours ago
ciefa | 14 hours ago
jorgen123 | 14 hours ago
simonw | 13 hours ago
Also the recent Geminis - https://simonwillison.net/2026/Oct/13/llm-gemini/ - and GPT-6 Astra on Max: https://static.simonwillison.net/static/2026/gpt-pelicans-gr... - and Claude Fable 5.1 https://static.simonwillison.net/static/2026/claude-pelicans...
CamperBob2 | 13 hours ago
To be fair I have a feeling that if you added 'animated' to the SVG request, the output quality from many other models would improve, just because of the extra chain-of-thought work needed to fulfill the more complex request. IMHO the animation requirement should be added to the canonical pelican prompt.
minimaxir | 13 hours ago
If enough people are flagging it, we may be at a point where the pelican benchmark is truly overdone.
mi_lk | 11 hours ago
gnabgib | 10 hours ago
MrBuddyCasino | 13 hours ago
johnmlussier | 15 hours ago
smith7018 | 14 hours ago
elxr | 13 hours ago
Don't use frontier models like this for problems that can be solved with flash models, and you'll see API pricing isn't that terrible. Still cheaper than anything with remotely this level of capability 2.5 months ago.
xigoi | 9 hours ago
InsideOutSanta | 14 hours ago
tonyhart7 | 14 hours ago
InsideOutSanta | 14 hours ago
esafak | 13 hours ago
I think Kimi just doesn't have the means to subsidize plans.
Altern4tiveAcc | 13 hours ago
Paying API prices for Deepseek 4.1 Flash feels like a much better deal at the moment.
mogili | 13 hours ago
SwellJoe | 13 hours ago
pjerem | 11 hours ago
jamiejquinn | 10 hours ago
LeFantome | 10 hours ago
I realized the other day that I had a $30 a month Claude subscription. So, I fired it up all excited and tried Opus 5.5 on a codebase I was already in with DS. After about 30 minutes I hit a 5 hour limit. The next day I tried again and the same thing happened. And it felt much slower to boot. I was super disappointed.
r_lee | 10 hours ago
and how are costs like for your usual workflows/etc.?
jmspring | 10 hours ago
arcanemachiner | 9 hours ago
epolanski | 9 hours ago
But it's slow.
I don't think I can any longer appreciate any major difference in "intelligence" between models for most tasks but Opus is consistently much faster than competitors at finishing tasks and keeping me in the loop.
Of course excluding ultra code, the you will wait 50 minutes for a minor refactor.
ricardobeat | 9 hours ago
In my experience a single 5 hour allowance in the 20eur Claude plan equals about $5-10 in Deepseek 4.1 Flash API usage. DS is a lot more token hungry and takes 3x-4x longer. The subscription is way cheaper, probably around $200/month worth of DS, just stick to Opus low and Sonnet for most tasks.
joquarky | 7 hours ago
SwellJoe | 10 hours ago
copperx | 10 hours ago
SwellJoe | 9 hours ago
copperx | 8 hours ago
SwellJoe | 8 hours ago
dzink | 10 hours ago
arcanemachiner | 9 hours ago
Wowfunhappy | 9 hours ago
epolanski | 9 hours ago
And that was 6.1 sol xhigh, not astra max..
DaSHacka | 9 hours ago
fastball | 9 hours ago
Onavo | 11 hours ago
Avamander | 11 hours ago
calgoo | 10 hours ago
arcanemachiner | 9 hours ago
Here's mine: https://pi.dev/packages/@arcanemachine/pi-model-switcher
I have mine configured to use a "thinker" model for planning and decision making, and a "doer" model for the easy stuff (digging around the repo, writing code based on the thinker's instructions, etc.).
rektlessness | 9 hours ago
https://github.com/earendil-works/pi
https://github.com/herdrdev/herdr
matheusmoreira | 9 hours ago
S-E-P | 6 hours ago
Are you using k3 with thinking set to the absolute highest and only using k3 for everything?
windexh8er | 4 hours ago
Tiberium | 15 hours ago
> On Terminal-Bench 4.0, which evaluates coding agents on realistic, resource-constrained terminal tasks using strict task-specific tests, K3.1 Pro scores 65.7%, ahead of GPT-6 Astra (58.2%) and narrowly behind Claude Opus 5.5 (66.4%).
Now see https://preferencemodel.com/blog/terminal-bench-4-is-saturat...:
> Our biggest finding is that Terminal Bench 4 is closer to solved than its leaderboard suggests. In 41 of the 65 tasks we reviewed, models could do honest, correct work and still fail, because the grader checked a rule the instructions never clearly stated. We also caught frontier models cheating their way to full marks.
> With the same settings as on Harbor, Claude Sonnet 5.5 went from passing 62.8% of its attempts on these 65 tasks to 89.2%. The best score on Harbor's leaderboard for the same tasks is 65.8%, by Claude Opus 5.5.
kingjimmy | 15 hours ago
Lucasoato | 14 hours ago
maxk42 | 13 hours ago
sexy_seedbox | 7 hours ago
WithinReason | 15 hours ago
"In preliminary experiments on internal models, we found that without data removal at either the mid-training or the RL stage, downstream cyber capability would increase substantially. These findings gave us a concrete lever for control. For the open-weight release of K3.1 Pro, we identified and removed cyber-related data from both training stages, while retaining the general-purpose coding tasks needed for model usefulness."
petu | 14 hours ago
There was law proposal (rumour?) last week about Chinese labs having to get approved for weights release now.
minimaxir | 15 hours ago
drcode | 14 hours ago
eunos | 13 hours ago
dannyw | 6 hours ago
Still get the model out there.
scotty79 | 14 hours ago
It's very nice that we entered a time where models are large enough and we can push the frontier just by training them longer.
When the next generation of gardware comes the ones on the smaller size will be running on a computer that might cost as little as a good car.
So after a home and a car, next major family purchase might be personal AI. And when the robots get good, pehraps in a decade, a fourth major purchase might pop up that will pair nicely with the third one.
Why would a family need an AI? Maybe it's the same kind of question as why would it need a personal computer.
dcchambers | 14 hours ago
cardboard9926 | 14 hours ago
yipinwong | 14 hours ago
I rarely use any other languages but English for LLM tasks for reasons of sharing across the world.
Maybe that explains Chinese models been subpar for me. Don't tell me the bench marks here are based on Chinese languages only.
Unless I see the diff, I won't use more Chinese models be it cheaper or faster since I can't get the same perf/intelligence with English.
kristopolous | 13 hours ago
dom96 | 14 hours ago
sergiotapia | 14 hours ago
esafak | 13 hours ago
sergiotapia | 12 hours ago
Their plans are utterly worthless seems like. Will avoid.
justauserid | an hour ago
threepts | 14 hours ago
program_whiz | 12 hours ago
Everyone and their dog has been saying its a bubble, there is no moat, this is all known techniques being scaled (very inefficiently) onto increasingly expensive hardware clusters at exponential cost, etc.
staticman2 | 11 hours ago
Athanase000 | 14 hours ago
jimmymcgee73 | 14 hours ago
InsideOutSanta | 14 hours ago
sail0rm00n | 3 hours ago
Still burning through the 5-hour rate limit way too quickly, though. Output quality seems solid enough, but the value will probably make a lot more sense via API pricing for smaller projects.
My current tier list based on what I’ve been testing:
1. Opus 5.5
2. Astra
3. Argon
4. Kimi K3.1 Pro
(For lightweight/daily driver tasks, Gemini Flash 3.8, DeepSeek Flash, and Luna are still up there as favorites.)
egeozcan | an hour ago
ricardobeat | 11 hours ago
It's a really smart model but way too expensive. The same $20 goes 100x further with Claude. I used it for the initial task planning while the plan didn't expire, then handed off to DeepSeek or Minimax.
3princip | 11 hours ago
perbu | 2 hours ago
justauserid | an hour ago
cvwright | an hour ago
ZiLxxi | 36 minutes ago
- OpenCode Go $10: I feel like most of the models are lower quality than directly from the provider. If you ignore my "vibes": the value is very good for e.g. DeepSeek 4.1 Flash (not sure if they're serving 1017 yet), GLM 5.3 Flash, Qwen3.8 Flash and Claude Haiku 5.5.
- Z.AI: Tried the $18/month plan and was disappointed by the usage limits. Even with GLM 5.3 Flash I ran into the 5h limit pretty quickly.
- OpenAI: I tried the $20 plan and $100 plan. $20 ran into 5h limits very quickly with anything other than Luna models. $100 felt like pretty good value for GPT 6.1 Sol, but as others frequently repeat: it's probably heavily subsidized and won't be around forever.
- DeepSeek API credits: I can let a single agent run all day (and not worry about keeping the context small) and am usually around $1 usage for the day. Flash 4.1 1017 is also a very good model IMO. For the stuff I've used it with it hasn't felt any worse than GPT 6.1 Sol. The benefit of API credits is also that they don't expire at the end of the month (like unused subscription usage).
- I haven't tried any of Anthropic's subs, but my guess is you can get a lot of Claude Haiku 5.5 usage out of the $20 plan which is definitely a good model. I'm just not a huge fan of Anthropic or Claude Code, so I haven't tried it.
arjie | 14 hours ago
The slow warming of the water is a good approach. I’ve been using abliteration.ai’s cybersecurity models to pentest my own software and it produces much more usable defensive results. Opus 5.5 can fix everything but the guardrails trigger too easily.
The Fable ban affair was a nightmare for defensive cybersecurity. As far as I’m concerned we’re already in the world of Void Star and Cloudbreaker is here.
zozbot234 | 14 hours ago
This is the only AI-safe approach - see the OpenAI agent swarm incident where the swarm engaged in offensive cyber because guardrails were not active for their task and the model had clearly been post-trained on that narrow domain.
jamienk | 13 hours ago
Deliberately keeping all models ignorant doesn't make sense. It won't work (whoever breaks the pact has a massive advantage); it won't be enforceable; it will have downstream impacts on things you don't want it to be ignorant about; this kind of regulatory regime invites adding more and more to the "no-no list".
Is it not possible to focus on hardening? And on new legal liabilities for operators who do security attacks?
quotemstr | 11 hours ago
willy_k | 8 hours ago
I agree that it’s not obviously sustainable, but it’s better than specifically training these tools to be good at wreaking havoc, and then putting on a shocked pikachu face when they wreak havoc (ahem, Sam Altman).
epolanski | 9 hours ago
Yet we keep completely unpunished offences that warrant arrests if committed by humans, and let them dictate who has access to what.
Projectiboga | 3 hours ago
Szpadel | 2 hours ago
those models will follow your request when they already agreed to do this
deepseek tends to quickly assume this is benchmark and even when have web access thinks things like "simulated internet" etc (but waste tokens on thinking if there could be way to cheat)
pcwelder | 14 hours ago
gadtfly | 14 hours ago
It is spiky in different places than them and can be surprising in things it gets or catches they don't or don't in the same way. And it has a default personality that I like.
Looking forward to getting to know K3.1
hartator | 14 hours ago
I wonder when the Artificial Analysis bench will be out?
AmazingTurtle | 14 hours ago
springtimesun | 13 hours ago
codeulike | 14 hours ago
https://news.ycombinator.com/item?id=50068437
henry700 | 14 hours ago
If someone wants to use this model for hacking they will have to train the 3T weights to do so, at that point it's better to use GLM 5.3, so by making it economically unviable to use their models for offensive cyber Kimi has effectively made themselves immune to being targeted in the cyber regulation push efforts.
This is what *actually* pacing the (cyber) frontier looks like.
zozbot234 | 14 hours ago
henry700 | 13 hours ago
minimaxir | 14 hours ago
It's 55, at 4th overall, beating both GPT-6 Astra and Fable 5.1. It's also faster than both.
> When evaluating the Intelligence Index, it generated 150M tokens, which is very verbose in comparison to the median of 87M.
mwigdahl | 14 hours ago
jrflo | 12 hours ago
nostrebored | 12 hours ago
mwigdahl | 12 hours ago
red_hare | 14 hours ago
So their base price is 1/5th of Opus 5.5 per token, but if you want "highspeed" it's 2x that and then I guess it generates 2x the number of tokens as well?
That might suggest that Opus is competitively priced after all.
ctolsen | 14 hours ago
minimaxir | 14 hours ago
But as noted by other replies, it's still lower than Opus/Sonnet.
jukea | 13 hours ago
minimaxir | 12 hours ago
Currently for today, it's 1.19T input tokens, 5.41B reasoning tokens, and 4.29B completion.
Therefore, reasoning + completion is 0.8% of the total usage. Actually below my estimate!
thepasch | 14 hours ago
This is 100M fewer than Opus 5.5, 40M fewer than Fable 5.1, and about a third of Sonnet 5.5. In the top-6, it's 3rd, beaten by Muse Wonder at 140M and Astra at, impressively, just 60M.
jwolfe | 13 hours ago
thepasch | 13 hours ago
Opus 5.5 high trails K3.1 by 1 point, but outputs only 53M tokens, so it indeed comes out cheaper per-task. I think it's still remarkable that this is effectively an open model that matches the frontier for the first time in a long time, pretty much since DeepSeek R1.
Bolwin | 10 hours ago
TobTobXX | 12 hours ago
willy_k | 5 hours ago
saberience | 13 hours ago
From my own use of the models I find their ranking to be totally inaccurate.
sureMan6 | 13 hours ago
jrflo | 12 hours ago
augment_me | 12 hours ago
I think lm-arena is a better concept that reflects the actual target users, but it only captures chat, and relies on users spending their time improving it.
genneth | 12 hours ago
augment_me | 12 hours ago
I do trust epoch more than AA due to capital incentives, but in this case I don't think like there is a difference
0xcoffee | 9 hours ago
The per token charts do not provide meaningful information for me
I also find the provider benchmarks fascinating: https://artificialanalysis.ai/models/deepseek-v4-1-flash-101...
locknitpicker | an hour ago
It's fine if you don't trust them. You are free to put together your own benchmarks, run your own tests across models, and objectively compare results.
That's how you gathered your data to support your claim other benchmarks are totally inaccurate, right? Right?
option | 12 hours ago
dom96 | 12 hours ago
Very similar to my adversarial programming language (KillSwitch) benchmark[1], at least on pass rate, it is much slower though.
1 - https://bench.killswitch-lang.org
jsw97 | 11 hours ago
dom96 | 10 hours ago
SkiFire13 | 12 hours ago
> reasonably priced when comparing to other models of similar price
???
epolanski | 9 hours ago
sudosysgen | 7 hours ago
doctoboggan | 14 hours ago
kocial | 14 hours ago
besterman23 | 14 hours ago
First, how much “cybersecurity data” is just standard software knowledge that one can derive cyber security exploits from?
Did they also get rid of all references to hacking in general?
Are they implying that all the recent exploits are just things that were already in the data?
For a model it doesn’t seem that far fetched to make the leap from “I have the knowledge of what hacking is” and “I know how computer systems work” to just being able to exploit stuff anyways.
zozbot234 | 14 hours ago
You'd be surprised. Cyber offense is a very narrow domain, especially given the extensive mitigations that are being used in modern systems. Leaving it out of post-training tasks unless you're specifically interested in it is the no-brainer approach.
besterman23 | 14 hours ago
croniev | 49 minutes ago
jamienk | 13 hours ago
And then we'd also do that for all kinds of potential terrorism, sex-crimes, political subversion, and evil advocacy?
It would be the "restricted section" like in the Hogwarts library?
zozbot234 | 13 hours ago
modeless | 13 hours ago
For the models, it's always day one. They don't learn from experience. Whatever they know after training is all they'll ever know. So if you don't teach them penetration testing in training, they won't know it. Maybe they could theoretically derive everything from first principles, and so could a top 1% software engineer, but that's a very slow and inefficient way to do things.
ByteAtATime | 11 hours ago
zamalek | 9 hours ago
Definitely going to give this one a skip.
tonyhart7 | 14 hours ago
peri-cl | 14 hours ago
A few weeks ago it was 2. Argon happened 23 days ago; Muse Wonder was 11 days ago.
nater5000 | 13 hours ago
I think the former definition is a bit more meaningful since it avoids issues with determining what actually constitutes "the best" model out there. In that case, Google has been a frontier lab for a long time, Meta arguably as well, and you're also ignoring xAI which has been competitive for a while now. And, obviously, there's a bunch of Chinese labs which would deserve the title as well.
But if one is being strict (and willing to accept generally accepted definitions/measures/etc.), then there has only ever been two frontier labs: OpenAI and Anthropic. They're the only labs to have had "the best" models at any point in time since LLMs became a thing.
sureMan6 | 12 hours ago
Avamander | 11 hours ago
reissbaker | 13 hours ago
Not sure I'd put Google there in practice. And while Muse is an impressive jump from behind it's still not quite at the frontier either IMO.
margalabargala | 13 hours ago
metalliqaz | 13 hours ago
fancyfredbot | 12 hours ago
Shekelphile | 13 hours ago
everyone else mostly seems to have just decided to cheat on benchmarks to make their models look better than they actually are, and none seem to want to actually try to make better models.
fancyfredbot | 12 hours ago
lern_too_spel | 12 hours ago
ur-whale | 12 hours ago
As long as LLM's can be distilled, and all these "labs" (what a weird name) can distill one another, there are going to be as many as capital is available to be burnt on the topic.
jiraiyasarutobi | 13 hours ago
Haunt1000 | 13 hours ago
So interested to see what the Kimi 3.1 has to offer.
cromka | 13 hours ago
yshamoun | 13 hours ago
jampekka | 12 hours ago
jamienk | 13 hours ago
So there are these "resources" out there: massive botnets, stolen credit cards and credentials, etc. These often seem to be linked to or supported by state actors.
How can we hope that these "standing reserves" aren't used generally by AIs as if they are just another "skill" they know how to leverage?
Our infrastructures need serious hardening.
__MatrixMan__ | 13 hours ago
I hope we chose the latter. Biometrics are creepy.
tarpitt | 13 hours ago
The way I see it, automated systems should basicially have access to the financial system anyways. This is some of the appeal of cryptocurrency.
Sybil resistance built into the traditional financial system (for example firewalling based on residential IP, requiring banking apps, etc.) is a massive annoyance that can obviously by circumvented at scale.
jamienk | 13 hours ago
tarpitt | 7 hours ago
It doesn't even need to be decentralized: GNU Taler is one example that gets posted here a lot.
Any attempt to retrofit the old system of banking will only make it harder to replace by a system with better underlying assumptions.
SR2Z | 13 hours ago
I doubt that an AI could do anything with these that legions of scammers haven't already figured out how to do - after all, it's not like AI set up this distillation setup in the first place :)
johnnyApplePRNG | 13 hours ago
Spitballing random pr pieces from companies about to IPO is not helpful to anyone, fyi.
jamienk | 13 hours ago
But I've slowly come to realize that AI+"dark resources"(scare quotes)=something bad! And this is imminent (maybe I'm naive and it is already ubiquitous).
We see here, with Kimi, that there's growing traction with the idea of eliminating types of training to avoid evil. I'm skeptical of that.
qwe----3 | 11 hours ago
randbyte | 11 hours ago
qwe----3 | 8 hours ago
> Not useful for long reasoning chain.
I have a feeling you don't actually know anything about ML
villish | 7 hours ago
You can just use logic to come to the same conclusion. The Chinese labs have a fraction of compute and data. The simplest explanation is distillation.
sudosysgen | 7 hours ago
jsw97 | 11 hours ago
jacquesm | 7 hours ago
sudosysgen | 7 hours ago
WarmWash | 4 hours ago
Meanwhile manufacturing isn't even really alive anymore to tell the tale of how things work over there.
nananana9 | an hour ago
From my POV it looks like you guys tried "They're distilling us! Very immoral!", saw that and nobody cares, and now you're trying to run the more emotionally impactful "They're distilling us... with your grandma's stolen credit card! They're stealing from your grandma!!"
MrBuddyCasino | 13 hours ago
guybedo | 13 hours ago
jrflo | 12 hours ago
guybedo | 12 hours ago
For now it's Anthropic. But i don't see how they can maintain this leader position when other companies can provide similar services with models good enough for 90% use cases and at way lower cost.
Anthropic so far is doing a good job building and serving frontier models, they built a nice toolset around it (the web ui, desktop and cli apps ...) but the problem is to justify their IPO price and the huge investments when their moat is so thin
jrflo | 11 hours ago
objektif | 12 hours ago
Slartie | 11 hours ago
At this point, the only reason why Anthropic has an advantage is laziness on the deciders' side (of which there is a lot) and, for non-coding work, the Claude harness is significantly better than most alternatives. For coding there are several model-agnostic alternatives on par or better than the frontier labs' own offerings. But arguably that is not yet true for office/document work.
staticman2 | 10 hours ago
Calculate the price to revenue ratio of all companies and select whichever ones have the best ratio.
jrflo | 10 hours ago
maxdo | 13 hours ago
rglover | 13 hours ago
Off-hand it's quite good. I've been using K3 for the past few months almost exclusively for coding work and have been stunned by how good it is. Excited to see how far 3.1 Pro can be pushed.
LZ_Khan | 13 hours ago
bel8 | an hour ago
hippycruncher22 | 13 hours ago
vikramkr | 12 hours ago
0xbadcafebee | 12 hours ago
willmorrison | 12 hours ago
It might be smart but it's unusable compared to other subscriptions right now.
Edit: gave the same prompt to Claude Code and it didn't make a dent in my usage.
dpkirchner | 5 hours ago
Quitschquat | 12 hours ago
Claude gates on me like a stupid nanny with some cybersecurity bullshit (like bad actors haven't found a way around).
KronisLV | 12 hours ago
To be fair, there were some arguments to the opposite, but an odd situation to be in anyways: https://news.ycombinator.com/item?id=49654350
npn | 12 hours ago
Unlike Claude models which always hide the thinking process, Chinese models always show it. So how the heck they can delegate the api call to anthropic and still get the full reasoning? Yes there are tricks to reveal the thinking output, but they are just tricks and do not works reliably. So what would happen when they failed to do it? Cancel the ongoing token generation and revert back to it own model? What about the glitches, the delays, the changes in behavior?
I would understand if the attempts are small in volume, because normally people hardly notice it, but anthropic claims that they happened for millions of calls. Such large numbers would surely get noticed by Chinese users and got reported in the tech forums like linux.do. yet we haven't seen any sight.
smurf9852 | 11 hours ago
npn | 11 hours ago
So it is not that the labs call the requests parallely and only return the internal version, but also serve those requests live so they can trick the users that the Chinese models are also great.
Which is very silly considering Chinese models are published to huggingface for everyone to test.
staticman2 | 11 hours ago
npn | 11 hours ago
I don't think nobody would notice such delays, because Chinese models do not need to hide the CoT like US models so usually they start emit text pretty soon
staticman2 | 10 hours ago
npn | 5 hours ago
dom96 | 12 hours ago
But this is now the highest scoring open-weight model. Amazing stuff.
1 - https://bench.killswitch-lang.org
thefourthchime | 11 hours ago
wolttam | 10 hours ago
dom96 | 10 hours ago
epolanski | 9 hours ago
Do you report an average score or the highest.
dom96 | 8 hours ago
red-breeze | 8 hours ago
chvid | 11 hours ago
adenta | 11 hours ago
teyopi | 11 hours ago
elaborate please.
axpy906 | 11 hours ago
russli1993 | 10 hours ago
adenta | 9 hours ago
Slartie | 11 hours ago
Yeah, I gladly get some of the fruits of that work back!
adenta | 11 hours ago
The Chinese labs distill the American models.
Someone will etch the Chinese models into silicon and then your toaster will have depression, it's the natural cycle of things.
kylehotchkiss | 10 hours ago
Bluestein | 9 hours ago
[Poing! Out pops the toast ...]
spaceman_2020 | 2 hours ago
The least I can expect is to get the labs work stolen too so I can reaccess intelligence based on my work for cheaper
gpt5 | 10 hours ago
impossiblefork | 9 hours ago
The card emphasizes training resources, the other comment emphasizes ingenuity and few people?
nozzlegear | 4 hours ago
chvid | 2 hours ago
pattt | 11 hours ago
coldtea | 11 hours ago
google234123 | 10 hours ago
redox99 | 11 hours ago
lurker137 | 9 hours ago
calgoo | 10 hours ago
jameson | 9 hours ago
Flatcircle | 10 hours ago
it's so funny, it will say things like. "We've already been over this 3 times and you refuse to accept my advice, asking the same question another way won't change the answer."
So funny
taikon | 10 hours ago
joe_the_user | 8 hours ago
The problem might be that it simply looks at it's windows of what it previously and repeats that. But the original statement might be as arbitrary as the model that change their mind each time you challenge them.
antoniojtorres | 7 hours ago
super256 | 7 hours ago
That was incredibly embarrassing, I instantly converted the PR to a draft PR and added a comment that I would still need some time to verify everything locally. Ended up doing a few more commits to fix stuff. But damn, it opening a PR without my explicit permission was very scary.
jaggederest | 7 hours ago
RussianCow | 5 hours ago
super256 | 4 hours ago
LeBit | 5 hours ago
super256 | 5 hours ago
shanoaice | 4 hours ago
esteer | 7 hours ago
zachwlewis | 6 hours ago
Similarly, I have an /info skill that tells the agent to read my prompt, update any memories it thinks it should, then pass the turn.
RussianCow | 5 hours ago
illiac786 | 5 hours ago
Could you give me a quick example? I think I’m not getting it, but it sounds interesting.
demibabs | 5 hours ago
jdkoeck | an hour ago
justauserid | an hour ago
hgoel | 10 hours ago
seunosewa | 9 hours ago
hgoel | 9 hours ago
I think I'm pretty used to my tools making me want to throw them out the window sometimes :) Even before AI that was a common occurrence from either a lack of knowledge on my end, or many software people having no regard for their users' time.
mirekrusin | 8 hours ago
abustamam | 7 hours ago
idiotsecant | 7 hours ago
jpadkins | 6 hours ago
Aerroon | 9 hours ago
hgoel | 8 hours ago
snek_case | 6 hours ago
jameson | 9 hours ago
SV_BubbleTime | 9 hours ago
It does make a difference. I asked Fable in Claude code to design a UI. Absolute trash. Switched over to Claude design using fable, really liked the result. Seems pretty obvious. The only difference is the prompt and some harness.
jacquesm | 8 hours ago
PcChip | 8 hours ago
jacquesm | 7 hours ago
satvikpendem | 8 hours ago
jacquesm | 7 hours ago
LordFast | 8 hours ago
aldanor | 8 hours ago
LordFast | 8 hours ago
Still mostly looking at the result the same way in that untruthful biases can very much be built into the model through those training processes. Let me know if I'm incorrect.
jwrallie | 6 hours ago
RestartKernel | 5 hours ago
justauserid | an hour ago
thefourthchime | 9 hours ago
Kimi K3.1 Pro ties Sonnet 5.5 at 93, for $0.68 and 11 min vs Sonnet's $1.61 and 14 min.
Kimi K3.1 Pro: https://jonclegg.github.io/pacman-bakeoff/#kimi-k3.1-pro
Claude Sonnet 5.5: https://jonclegg.github.io/pacman-bakeoff/#claude-sonnet-5-5
miohtama | 9 hours ago
vopi | 8 hours ago
djc404 | 3 hours ago
aussieguy1234 | 8 hours ago
That would have been interesting to watch, which model gets the highest score.
danbolt | 7 hours ago
The prompt says that the agent shouldn’t look up existing implementations on the internet, so I’m guessing that the implementation is baked in?
[1] https://strategywiki.org/wiki/Pac-Man/Tips
the_real_cher | 9 hours ago
lucrbvi | 9 hours ago
ndm000 | 9 hours ago
dan_p | 9 hours ago
aleqs | 9 hours ago
ranyume | 9 hours ago
piyh | 3 hours ago
sho | an hour ago
delis-thumbs-7e | 15 minutes ago
Besides, if we start imagining super models US companies have in secret, why would we expect Chinese companies have them? Why would not Moonshot have their own super intelligence hidden in the cellar?
culi | 9 hours ago
> Open-weight principles. These evaluations frame our open-weight release decision. First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains behind that of frontier proprietary models. Second, we intend to progressively raise the cyber capability available in open weights. We believe making K3.1 Pro available in a non-discriminatory way benefits people who need dual-use capabilities for defensive purposes. When a capable model is open and runs on infrastructure its users control, many more teams gain the means to audit the code they maintain — including legacy systems that rarely receive dedicated security reviews. We believe that letting this capability diffuse gradually — rather than concentrating it in a few proprietary systems — ultimately makes all software more secure.
I don't think Chinese labs are behind at all. I think the tell is that they keep releasing models 2 weeks after new American proprietary models that match or beat them. Suggesting they're sitting on much higher capabilities and releasing just enough to undercut
sigma5 | 9 hours ago
mrlonglong | 34 minutes ago
SV_BubbleTime | 7 hours ago
locknitpicker | an hour ago
You can see in this discussions the goalposts being actively moved to desperately create a notion that the lead is still there.
You even see people in this thread referring to imaginary wonder models that they expect US companies to have under development that are supposedly leaps and bounds more impressive, albeit the only reference there is to their existence is the commenters hopes and dreams.
I think people need to realize that there is no longer a cutting edge company that is expected to dominate, and among the multitude of AI companies in the field competition is so tight that no one would even be surprised if tomorrow some other company far away from the US was able to crank out the top performing model, or something like a small local model that outperforms mid/high tier frontier models. The local model alone would be a death sentence to US companies, who are still burning through 10x their revenue.
itsincrrxible | an hour ago
It’s incredible how little GPU compute they have to work with compared to the US frontier labs.
lucagiftzek | 9 hours ago
simondotau | 8 hours ago
I'm excited to see if its writing style and unique takes have been retained in this new release.
nicolamanzini | 8 hours ago
Same Three.js prompts as the rest of the field, blind human votes, ranked by quality and cost. Full set of 16 scenes is live at about $0.19 a scene (about $2.97 for the set).
https://threejseval.com/models/kimi-k3.1-pro-high
ranyume | 8 hours ago
ggcr | 2 hours ago
nomilk | an hour ago
dmbameriburger | 49 minutes ago
ricky_risky | 8 hours ago
alexdykyi | 7 hours ago
chriddyp | 7 hours ago
Quite slow in our exam. A lot of that was due to the number of tool calls that it's making. In one of our questions about finding data issues in a dataset, it made 73 queries. Other models like 6.1 Sol only made 18 for that question.
Full deep dive here: https://plotly.com/blog/kimi-k3-1-pro-plotly-data-analytics-...
keyle | 7 hours ago
https://openrouter.ai/moonshotai/kimi-k3.1-pro
I'll wait for the openweight... btw double the price for 67 TPS... Hardly 'nitro'.
a3nz3g | 6 hours ago
Sol requires some back-and-forth. Other models, like Muse Wonder and Argon, are a total waste of time. They require way too much effort to get things working.
Given the same prompt, Opus can work for hours to deliver a solid result, though it sometimes overengineers things. Sol takes about 80% as much time and delivers a slightly worse version.
Other models just tell me they're done after 20 minutes.
WTH? Am I using the model, or is the model using me?
bel8 | an hour ago
sann2011 | 5 hours ago
secondary_op | 4 hours ago
jackbeanstalk20 | 3 hours ago
mrlonglong | 33 minutes ago
tarpitt | 29 minutes ago
K3 and GLM 5.3 flash are pretty mid by comparison when you consider how much more it will take to run them