A bit off topic, but I think I'm starting to get model fatigue. These come out 10x faster than new Javascript frameworks were coming out 10 years ago (at least new models are far easier to adopt).
There was a time when every new PC CPU coming out was a giant deal: "Guys have you heard about this new Pentium processor, it's incredible?"
But over time, more and more people got into the chip-making business, and the big players started releasing more and more chips. Now only the die-hard CPU trackers worry about every new CPU and exactly how it's better ... while everyone else just worries about "which CPU will be good enough at this moment".
Same for smartphones. There was a time of unlimited hype and secrery around new iphones. Who remembers the story of an iphone 5 prototype left a bar. Journalists were going crazy, people were signing petitions for Apple to not hunt down but instead forgive the employee that made such a grave mistake. People were offering millions to buy the prototype so they can brag they got the new phone 2 weeks before everyone else did.
Or who remembers the dancing disease of 1518, were people would stop what they are doing and start randomly doing the same dance. The lords? Out of their minds. The priests? Terrified the devil had taken hold of the flock!
I have come to believe that it was probably some tik-tok like hype trend of doing a fortnite dance while waiting in line for bread and communion. And the energy back then, like now, was off the charts.
Hype and memetic trend seeking encoded deep in human psyche.
Honestly, you don’t have to pay attention. What you do with models matters way more than the models themselves, and you don’t need frontier for the vast, vast majority of use cases
> 32B: Ranking among the top models in its class, 32B is our most powerful dense model, balancing capability, adaptability, and local deployability.
> 7B: The industry’s best-performing model under 10B combines strong software engineering and expert knowledge in a package small enough to run on a phone.
Fully open models really need to be a big part of the AI future. That includes all source code, open training data, how it's organized, fed to the model, processed, etc. Until that becomes a thing you're always going to be left wondering what exactly lies underneath the closed model you are using, leaving open the possibility for societal manipulation.
Why? Sure, I’d prefer it, too, but this is just another GNU/Linux vs. macOS situation: most of us would prefer the first, but actually get shit done on the latter.
we get shit done on the cloud with the former rather than the later
I personally find the analogy unconvincing, the UX dimension is completely different as I can use the same harness with any model; and the year of the linux desktop is coming soon (tm)
Why do you make the worse choice and not use what you would prefer to use? You have been able to "get shit done" on Linux for nearly 30 years. Have the courage of your convictions.
You could sidestep it by running non-permissibly licensed training data that you purchased through an LLM. Legal attitude so far seems to be that this is transformative as long as it's not 1:1. The question on whether or not the end result is copyrightable of course remains controversial and inconsistent, but that question is also fairly irrelevent. You don't get more libre than public domain.
That's a fair amount of computational and labor overhead mind you, as you'll need to verify and prune the quality of your mountain of synthetic data, but certainly possible.
Though this assumes the legal system is a rational actor playing by the set of rules it claims to. In fact, I highly suspect you could get very unlucky and get an unfavorable ruling against you, because you stepped on a big pile of money's toes in the process of doing this.
Eventually we'll just construct 100% synthetic training data that can reliably reproduce pretrains and fine tunes.
The first broadly useful fully open source models will do this.
We already have open data / open code / open weights for some domain-specific cases, such as audio models trained on large open datasets, eg. Tacotron / LJSpeech from waaay back in the day, though that is certainly not SOTA anymore.
Distillation could possibly be considered an early case of this as raw AI outputs are themselves not copyrightable unless humans enrich, filter, or transform them. Granted, that does not handle the cases where the outputs are sufficiently similar to copyrighted original works.
But how much of that synthetic data still ultimately derives from non-open sources? You'd still have to ask what a clean room implementation ultimately is, depending on how granular or aggressive a large publisher wanted to get about it.
That said, I don't necessarily disagree with you. Talkie[1] presents an interesting case for it being at least possible to do this entirely on public domain material.
But even that used Claude somewhere in the course of its training pipeline (it's listed as a contributor on their GitHub), so again, how granular you want to get with that is still a question.
Decentralized unstoppable storage, combined with decentralized unstoppable training, sorta like SETI for AI training. The seed of this tech already exists with IPFS and others like it.
We know (some? all?) of the big labs have skirted copyright laws at one point or another. Truly open models would just build on what is publicly available.
If the LLM/AI ecosystem starts actually needing some Person-To-Person (or maybe Agent-To-Agent?) payment system because things actually get smart enough to be useful autonomously, they're gonna need some way to send money/currency around. Depending on how banks will react to this need, we might see another return of digital currencies from the current winter.
Or, we just need to get this over with and declare any digital data findable via the internet to just be public property of everyone. Everything becomes public, besides stuff you keep locally, and there is no difference anymore, it's all just data anyone can use for whatever. A 1 year grace period for everyone to pull stuff off they don't want to be a part of this bright new open era, then we just scrap everything related to intellectual property, copyright and similar stupid stuff, and slap UBI on top of all of it for good measure.
I'd prefer to stay within the [hacker ethics](https://www.ccc.de/en/hackerethics), and protect private data. For non-private/personal data, sure. But individual people need their privacy protected.
Me too, I'm hacker ethics all the way, which is why I'm saying anything network connected should really realize the "All information should be free." dream, and then private data should be far away from the internet, on computers/drives not even connected to the internet. The whole E2E encryption is a ticking time bomb people rely to keep their data safe from others, but nothing that you don't physically have close to you can be truly secret forever, and even then it'll be hard.
I respectful disagree. I enjoy reading e.g. Asimov and well-executed journalism. And I completely respect the IP of those people who create these works.
That dead man took the risk of not earning much in his lifetime, to continue feeding his family even after his passing. You might as well ask why does someone acquire life insurance.
The current "lifetime of the author + X years" rules in effect in the United States still carry the same perverse incentive, though the incentive diminishes rapidly as X gets larger; with the very large value of X in effect today the perverse incentive is so small as to be effectively non-existent, but it's still there in theory.
Personally, I'd prefer a fixed term. I know enough independent authors making a living from selling their books that I'm willing to allow the fixed term to be large, like 50 years from date of completion of the work. (With a good definition of "completion" so someone can't cheat by editing a couple lines per year to keep something copyrighted indefinitely). The simpler the rule is, the easier it is to understand, and the harder it is to cheat it. The more complicated you make a rule, the more loopholes get found.
What you're slightly more realistically looking for here is for publicly available data to have a Fair Use exemption for certain uses, which is certainly something worth discussing.
I don't really get what you're suggesting. You give a 1 year grace period for Metallica to pull all its music off the Internet, but then as soon as I host some of their MP3s on my Wordpress blog it's "public property of everyone" from that point forward?
I hate to invoke Poe's law but, I've now flip-flopped like six times over whether this could be serious.
I think it is serious. In which case, I gotta say, it really seems like you didn't spend much time thinking about this. "A 1 year grave period for everyone to pull stuff off they don't want to be a part of" - How does that work when the Internet is already full of unauthorized reproductions, most of which people aren't even aware of? Even ignoring practical considerations, when literally everyone is basically stuck using the Internet for everything, this seems a bit unfair to anyone who isn't onboard, akin to The Onion's Google Opt-out Village. But there are so many practical issues with this, it would be easier to list the number of problems this doesn't have. You accidentally leak something to the Internet and it becomes commons? What happens when other people leak things to the Internet? How about revenge porn?
Not minor stuff that can easily be papered over, this literally reintroduces the problem of needing to care about the provenance of data again, in a way that can't be automated, which makes the whole thing entirely moot. All just to make training data for AI models easier to distribute?
I'm all for intellectual property reform, maybe even fairly radical. But this just seems like it wasn't thought out.
If this was satire, well, I took the bait. Oddly convincing despite being hard to believe.
Open models can be used/changed for social manipulation too, by anyone, which scares a bunch of people, as opposed to the dark pattern manipulation from Big Ai/Tech
Other than open training data (currently legally impossible), all of this holds for basically every major Chinese-made model. They not only open the weights but publish detailed methodology papers alongside the models in arXiv and even open source the code.
I kind of assumed all the K2 names were puns. K2 is quite tall, so to get to the top of it you have to be really good at hill climbing. Anyway it’s a pretty well known mountain so I don’t think anyone can call dibs on it.
The comparisons with other models here are odd.. the other models change depending on the task. It would be far more useful to at least compare against the more recent open models (DS4Flash/GLM53Flash/Qwen38).
It is great to see another player introduce a fully open stack. Nvidia's Nemotron is the only other prominent one I know of.
All that said, the headline claims do not match the self-reported performance. For example, the dense 32B model is significantly behind Qwen3.8 27B (chart towards the bottom of https://ifm.ai/blog/k2). Gemma4 31B is not in the comparison set. This is the most important sweet spot for self hosted open-weight models today and real competition here will be very welcome.
They have the 32B listed as "stage 1" with the note "final checkpoint to be released." So, not finished yet. Not sure why you'd release it if it's not finished, but that's the explanation.
I just looked on Huggingface.co, and the training data is there.
For example, 3.3 Tbyte for code reasoning, 4.5 Tbyte for mathematical reasoning, 8.4 Tbyte of pre-train behaviors, and so on.
I did not compute the sum of the dataset sizes, but it appears to be some tens of Tbyte. Nonetheless, I assume that this amount of training data is more than an order of magnitude less than what OpenAI, Anthropic and the like have used, which must have been at least many hundreds of Tbyte, but more likely several thousands of Tbyte of data.
fair, I stand corrected on the data being there. the part I'm still holding judgment on is whether the full training recipe ships too, not just the raw bytes.
Their previous model, K2 Think V2, was release with fully open training data and recipe, so I would imagine that they are committed to that, but yeah, the repos for this new model are still just placeholders.
I find it funny that while these releases are a technological miracle, the charts in the doc use tiny fonts and are hard to read. Goes with the idea that coding might be solved, but taste isn't.
Accessibility isn't "solved," but there are certainly standards for things like color contrast. Maybe inbetween taste and coding there are better targets still being missed.
In fact, automated a11y checks and tooling is quite advanced nowadays and tragically underutilized by web developers. Now that we have llms to scale all the shitty code of front-end devs at startups, I feel increasingly hopeless about things ever improving
My quick review of the 3.7B model (because I was interested) is that it's not to be trusted for coding.
It failed my basic test I like to ask models and generated incorrect code. When prompted about the bug, it preceded to start hallucinating non-existent APIs. After doing that it got caught in a loop trying to desk check the solution that didn't work.
It's actually just a coding interview test that I liked to ask in the past. You can find it and others on leetcode.
The reason I personally like my question is because it's pretty close to some of the real world work we do. It's mostly mundane and easy to bang out, but really easy for someone to do a n log n solution where an n solution exists.
A good example (but not my question) would be something like
"I have a list of People objects with a `first` and `last` name. Write a function which groups together all the People with the same last name in `your language of choice`"
Now a days? No. It's actually getting to be a bad question because they all push out about the exact same answer.
But much earlier they did and, apparently, these really small models still do. At this point it serves as more of a smoke test for me. Success means little, failure means a lot.
Not sure a model that small is really supposed to be used for any real coding. At that size you're usually using the model to do simple tasks like summarization.
To be clear, the question wasn't a complex one. It was more on the level of "could I use this for a fast inline coder" IE, single somewhat simple function question.
I wouldn't have dreamed to use this as an agent model.
7B models of the past have been able to pass this question. I've not tested it on a 4B model until now.
I'm glad to see some development in the space of "truly open" models that share training data and other recipes. As the costs for hardware fall over time (hopefully), we should see more possibility in fine-tuning and developing software to inspect the source training material.
Some other open models I'm aware of:
- OLMo
- Apertus
- Soofi
- OpenEuroLLM
- llm-jp
OLMo is perhaps the most famous, and their Dolma training corpus has been reused in other projects. It looks like the K2 training materials haven't been released yet, but I'm interested to see what they did for training "long-horizon agentic tasks". I'm aware of SWE-smith + SWE-gym but I'm guessing there's a lot more out there now.
I'm no expert, which is part of why these projects excite me. I'm hoping they can be good projects to learn from as well.
While the open approach is commendable. The 32b and 36b models are inferior to qwen 3.8 27b, at least according to benchmark numbers. I would have liked to have seen both compared to 27b, not only the dense one. Also would have liked to see more coding benchmarks in the full table.
piinbinary | 11 hours ago
kelseyfrog | 11 hours ago
JSR_FDED | 9 hours ago
wuhhh | 11 hours ago
hungryhobbit | 10 hours ago
But over time, more and more people got into the chip-making business, and the big players started releasing more and more chips. Now only the die-hard CPU trackers worry about every new CPU and exactly how it's better ... while everyone else just worries about "which CPU will be good enough at this moment".
I think models are on that same arc.
pantelisk | 8 hours ago
Or who remembers the dancing disease of 1518, were people would stop what they are doing and start randomly doing the same dance. The lords? Out of their minds. The priests? Terrified the devil had taken hold of the flock! I have come to believe that it was probably some tik-tok like hype trend of doing a fortnite dance while waiting in line for bread and communion. And the energy back then, like now, was off the charts.
Hype and memetic trend seeking encoded deep in human psyche.
dgellow | 9 hours ago
kamranjon | 11 hours ago
gs17 | 11 hours ago
esafak | 11 hours ago
wmedrano | 11 hours ago
https://huggingface.co/collections/IFM/k2-horizon
sottol | 11 hours ago
verdverm | 9 hours ago
sottol | 11 hours ago
https://ifm.ai/k2/
375 A23B, 36 A4B, 32B, 7B, 3.7B, 0.9B variants.
> 32B: Ranking among the top models in its class, 32B is our most powerful dense model, balancing capability, adaptability, and local deployability.
> 7B: The industry’s best-performing model under 10B combines strong software engineering and expert knowledge in a package small enough to run on a phone.
jjordan | 11 hours ago
trvz | 11 hours ago
homarp | 11 hours ago
zufallsheld | 11 hours ago
didibus | 10 hours ago
verdverm | 9 hours ago
I personally find the analogy unconvincing, the UX dimension is completely different as I can use the same harness with any model; and the year of the linux desktop is coming soon (tm)
eikenberry | 7 hours ago
kibae | 11 hours ago
ux266478 | 11 hours ago
That's a fair amount of computational and labor overhead mind you, as you'll need to verify and prune the quality of your mountain of synthetic data, but certainly possible.
Though this assumes the legal system is a rational actor playing by the set of rules it claims to. In fact, I highly suspect you could get very unlucky and get an unfavorable ruling against you, because you stepped on a big pile of money's toes in the process of doing this.
alightsoul | 9 hours ago
Are LLMs what we need to make all data public domain? This way it could be used for that purpose
echelon | 11 hours ago
The first broadly useful fully open source models will do this.
We already have open data / open code / open weights for some domain-specific cases, such as audio models trained on large open datasets, eg. Tacotron / LJSpeech from waaay back in the day, though that is certainly not SOTA anymore.
Distillation could possibly be considered an early case of this as raw AI outputs are themselves not copyrightable unless humans enrich, filter, or transform them. Granted, that does not handle the cases where the outputs are sufficiently similar to copyrighted original works.
waffleiron | 10 hours ago
chaosharmonic | 10 hours ago
That said, I don't necessarily disagree with you. Talkie[1] presents an interesting case for it being at least possible to do this entirely on public domain material.
But even that used Claude somewhere in the course of its training pipeline (it's listed as a contributor on their GitHub), so again, how granular you want to get with that is still a question.
[1] https://talkie-lm.com/chat
jjordan | 10 hours ago
Decentralized unstoppable storage, combined with decentralized unstoppable training, sorta like SETI for AI training. The seed of this tech already exists with IPFS and others like it.
We know (some? all?) of the big labs have skirted copyright laws at one point or another. Truly open models would just build on what is publicly available.
alightsoul | 9 hours ago
embedding-shape | 9 hours ago
idiotsecant | 3 hours ago
embedding-shape | 9 hours ago
chme | 9 hours ago
embedding-shape | 7 hours ago
dotancohen | 8 hours ago
idiotsecant | 3 hours ago
dotancohen | 3 hours ago
ceejayoz | 2 hours ago
rmunn | an hour ago
Personally, I'd prefer a fixed term. I know enough independent authors making a living from selling their books that I'm willing to allow the fixed term to be large, like 50 years from date of completion of the work. (With a good definition of "completion" so someone can't cheat by editing a couple lines per year to keep something copyrighted indefinitely). The simpler the rule is, the easier it is to understand, and the harder it is to cheat it. The more complicated you make a rule, the more loopholes get found.
idiotsecant | 8 minutes ago
jrm4 | 7 hours ago
tshaddox | 7 hours ago
jchw | an hour ago
I think it is serious. In which case, I gotta say, it really seems like you didn't spend much time thinking about this. "A 1 year grave period for everyone to pull stuff off they don't want to be a part of" - How does that work when the Internet is already full of unauthorized reproductions, most of which people aren't even aware of? Even ignoring practical considerations, when literally everyone is basically stuck using the Internet for everything, this seems a bit unfair to anyone who isn't onboard, akin to The Onion's Google Opt-out Village. But there are so many practical issues with this, it would be easier to list the number of problems this doesn't have. You accidentally leak something to the Internet and it becomes commons? What happens when other people leak things to the Internet? How about revenge porn?
Not minor stuff that can easily be papered over, this literally reintroduces the problem of needing to care about the provenance of data again, in a way that can't be automated, which makes the whole thing entirely moot. All just to make training data for AI models easier to distribute?
I'm all for intellectual property reform, maybe even fairly radical. But this just seems like it wasn't thought out.
If this was satire, well, I took the bait. Oddly convincing despite being hard to believe.
ignoramous | 9 hours ago
cute_boi | 10 hours ago
__MatrixMan__ | 10 hours ago
Sure there are all kinds of problems with that situation. But it still demonstrates that they can be coerced: play nice or don't play at all.
verdverm | 9 hours ago
https://allenai.org/olmo
Open models can be used/changed for social manipulation too, by anyone, which scares a bunch of people, as opposed to the dark pattern manipulation from Big Ai/Tech
culi | 6 hours ago
theplumber | 5 hours ago
jon9544hn | 11 hours ago
sottol | 11 hours ago
reasonableklout | 9 hours ago
luckydata | 11 hours ago
afzalive | 11 hours ago
bee_rider | 11 hours ago
Topfi | 7 hours ago
mmastrac | 11 hours ago
cogman10 | 9 hours ago
There is, for example, no Qwen3.8 7B.
It is odd to me, though, that they didn't run the same benchmark suite for the various quants.
a11r | 11 hours ago
All that said, the headline claims do not match the self-reported performance. For example, the dense 32B model is significantly behind Qwen3.8 27B (chart towards the bottom of https://ifm.ai/blog/k2). Gemma4 31B is not in the comparison set. This is the most important sweet spot for self hosted open-weight models today and real competition here will be very welcome.
xienze | 10 hours ago
The 7B does look very, very good however.
WithinReason | 10 hours ago
bluejay2387 | 9 hours ago
baron3dl | 10 hours ago
uniclaude | 11 hours ago
prometheus1992 | 10 hours ago
villish | 10 hours ago
luciana1u | 10 hours ago
dakolli | 10 hours ago
adrian_b | 9 hours ago
For example, 3.3 Tbyte for code reasoning, 4.5 Tbyte for mathematical reasoning, 8.4 Tbyte of pre-train behaviors, and so on.
I did not compute the sum of the dataset sizes, but it appears to be some tens of Tbyte. Nonetheless, I assume that this amount of training data is more than an order of magnitude less than what OpenAI, Anthropic and the like have used, which must have been at least many hundreds of Tbyte, but more likely several thousands of Tbyte of data.
luciana1u | 5 hours ago
lambda | 2 hours ago
* https://github.com/ifm-ai/xllm * https://github.com/ifm-ai/horizon-post-train
Their previous model, K2 Think V2, was release with fully open training data and recipe, so I would imagine that they are committed to that, but yeah, the repos for this new model are still just placeholders.
* https://mbzuai.ac.ae/news/k2-think-v2-a-fully-sovereign-reas... * https://github.com/LLM360/Reasoning360
cesarvarela | 9 hours ago
mzmzmzm | 7 hours ago
culi | 6 hours ago
TechSquidTV | 9 hours ago
edit: Tried signing up and using the internal playground. Holy shit thats fast.
cogman10 | 9 hours ago
It failed my basic test I like to ask models and generated incorrect code. When prompted about the bug, it preceded to start hallucinating non-existent APIs. After doing that it got caught in a loop trying to desk check the solution that didn't work.
dotancohen | 8 hours ago
cogman10 | 8 hours ago
The reason I personally like my question is because it's pretty close to some of the real world work we do. It's mostly mundane and easy to bang out, but really easy for someone to do a n log n solution where an n solution exists.
A good example (but not my question) would be something like
"I have a list of People objects with a `first` and `last` name. Write a function which groups together all the People with the same last name in `your language of choice`"
dotancohen | 7 hours ago
cogman10 | 7 hours ago
But much earlier they did and, apparently, these really small models still do. At this point it serves as more of a smoke test for me. Success means little, failure means a lot.
dotancohen | 4 hours ago
xienze | 7 hours ago
cogman10 | 7 hours ago
I wouldn't have dreamed to use this as an agent model.
7B models of the past have been able to pass this question. I've not tested it on a 4B model until now.
cogman10 | 6 hours ago
The first attempt with 7B the model got stuck in an infinite loop.
justin_ | 9 hours ago
Some other open models I'm aware of:
OLMo is perhaps the most famous, and their Dolma training corpus has been reused in other projects. It looks like the K2 training materials haven't been released yet, but I'm interested to see what they did for training "long-horizon agentic tasks". I'm aware of SWE-smith + SWE-gym but I'm guessing there's a lot more out there now.I'm no expert, which is part of why these projects excite me. I'm hoping they can be good projects to learn from as well.
kzrdude | 6 hours ago
https://huggingface.co/IFM/K2-Horizon-7B-Uno
throwawayffffas | 4 hours ago
RandyOrion | 23 minutes ago