We already know that LLM productivity is not real, just a self perception
This links to METR's early-2025 work. They've released a newer analysis from August 2025 that tells quite a different story. Note, that's still prior to the end-2025 models that changed a lot of devs' minds.
We've went through a few cycles of "ok, this model sucked but just wait for the next one", but in 2026 they actually started crossing the good-enough threshold.
It still depends on the type of the task, and code quality is meh, but there is a major difference between "hallucinated stackoverflow" of 2024 and "junior coder" of 2026.
Caught me as well. Models have improved much after 3.5/3.7, and much of my productivity increase comes not from a single project but from being able to work on 4-5 projects simultaneously and maintain 10 by myself
I kinda disagree with calling a proposal that is not an outright ban "LLM friendly".
If I were an AI hypester, I'd be positively gleeful at every one of these overzealous judgements passed towards projects, who try to impose limits, but not enough to pass the purity line.
People, who haven't formed strong opinions yet will see the outrage, check what the actual policy was supposed to be, see that it was really not nearly as bad as it was portrayed as, and will thus put less weight behind arguments raised by antis.
It's like astroturfing, but it's your own very enemies doing all the work, while you don't have to do anything.
Yep. I'm already checked out on most of the antis. They decry the loss of craft, their hobby and joy. And then follow up with no of course they haven't used it.
I wasn't around for the first "high-level languages" like FORTRAN, but I presume the guys who handwrote assembly and wired the memory themselves were just as salty about the coming onslaught of inefficiency and bad software.
I don't need to use something to pass judgement on it, especially when I can see its direct effects on my coworkers and peers. I'm getting really sick of this "you haven't even used it yourself" argument.
I worry about people who are heavily using LLMs who haven't overcome serious addiction before in their lives. They may not be able to recognize the feeling.
I'm getting really sick of this "you haven't even used it yourself" argument.
I kind of like it. It is an easy way for people to self-identify as "I didn't even take the time to try to understand the other side's argument.", making them easier to dismiss/ignore.
On the flip side, I generally ask that question when the person is making an assertion that, in my personal experience using LLMs judiciously, seems either completely false or “was true 16 months ago but not now”.
I don't really see how this is related. Having used it, not having changed my stance from being against it, I still think I could've judged it all the same just from observing the output.
It's a bit like reading bad fiction. Reading it is enough, I don't need to try writing it.
That said, I'm unenthusiastically ok with using it at work where it makes sense, but I prefer to keep my open source free from it. That might make me some capitalism apologist because I'd like my company to make money to pay me, but I don't see why most FLOSS development has to be be "improved" that way
I presume the guys who handwrote assembly and wired the memory themselves were just as salty about the coming onslaught of inefficiency and bad software.
Yeah, it was mostly real idiots like Edsger Dijkstra anyway. No need to listen to them. (/s, as I should hope is obvious.)
I wasn't around for the first "high-level languages" like FORTRAN, but I presume the guys who handwrote assembly and wired the memory themselves were just as salty about the coming onslaught of inefficiency and bad software.
And they were right to do so, things have gotten worse and worse. If AI is so good then why does everything made with it suck so bad? I don't have to use AI to be subjected to its poisonous fruits against my will.
I believe things had been getting worse already, especially since the advent of frameworks-of-the-week taking over the vast majority of webdev, and websites taking over the majority of human-computer-interaction. AI has simply accelerated the enshittification and the revealing of those without taste / fundamentals. Spotify was a broken POS before AI, and this post predates AI too. We're just accelerating the need for a reckoning with global software quality.
Perhaps, but many are salty about much more than the coming onslaught of inefficiency and bad software. People dislike LLMs for reasons that are generally not normally acceptable to consider, sometimes even in egalitarian open-source environments. Often these reasons are illegible and hard to verbalize, and can cause people to have to confront uncomfortable ethical inconsistencies within themselves.
On the other side there is "completionism", where FLOSS is "just a product" and its only goal is to always be better, faster, safer. Any fun to be had is in individually solving technical problems and requirements. Social bonds may happen, but colleagues are more coworkers than community. This is a "100% allglitches alltricks" TAS speedrun. The "we are apolitical", "we only care about the code" view.
This is a deep misunderstanding of the "completionist" project, or at least large parts of it.
The completionist project is profoundly political, dating right back to Stallman himself. The goal was to provide an operating system and a set of tools that preserved the freedom of end-users to do whatever they wanted with their computer. This is essentially an "anarchist" project, in a political sense: Devolve power to the people. Allow them to do anything they want with their hardware. And Stallman chose essentially anarchist means: Programmers would build GNU themselves, and give it away under a copyleft license, to prevent the commons from being enclosed.
Under the GNU Project's goals, nothing suggests that you couldn't just ask a benevolent Culture Mind to just implement the entire GNU system, document it extensively, and release it under copyleft. (Let's assume Culture Minds can legally apply copyleft to their own creations, based on the legal argument of "give the nice Rapid Offensive Units anything they want.") Boom, total end-user freedom, done in an afternoon.
A modest proposal: Since I'm mildly annoyed with people declaring that the completionists are apolitical, I'm going to propose an even more radical completionist project. To be clear: I do not necessarily endorse this project.
The goal: Every non-technical end user should not just have the hypothetical ability to modify their operating system and applications. Instead, we want to make this ability real, for every user. Minimum system requirements will be 32GB of GPU-like RAM and 500 GB/s of memory bandwidth, so this project will need to wait until after the next big RAM price crash. This system would include:
A kernel and applications, covering ordinary user needs.
An open-weight coding LLM that runs 100% locally in those 32 GB of VRAM. Aim for 200W when actually running, or less, if laptop GPUs get good enough.
Strong "rollback" guardrails and data protection, making it easy to recover from many (but not all) common mistakes. Maybe more advanced changes to the system core would require turning on "developer mode" and reading some warnings.
"Skill" documentation explaining to the LLMs how to make common changes to the user's system.
Simple application-building frameworks for GUI, web and CLI apps which are designed to be accessible to both humans and LLMs.
Pointers to programming tutorials so that users can actually build some understanding, too. And skills that explain to the LLMs how to walk users through code and architecture.
An end-user app store for sharing sandboxed vibe slop (but only if it preserves the "four freedoms"). I'm being mean describing it as slop, but I bet people would actually love it, if it was done well. Roblox for free software, basically.
Now, the user not only has the full source code to their system, they also have a 100% local coding agent that they can talk to and ask to customize things. With luck, the guard rails and data protection will mostly hold up. Freedom can sometimes be messy, and this version might get messy indeed!
Honestly, reading this is giving me some serious nostalgia vibes. It’s not the same, at all, but I learned to program on a Commodore VIC-20 and most of the programs I started with came from books that I took out from the public library. Clearly not the same, but… spiritually I see it! The system was mutable. It didn’t take much study for me to be able to start understanding how the games from those books worked and then riff on them to make my own.
I’ll toss point 8. into the ring to see if you like it: a clever way for users to share their customizations with each other. I’d say git but there’s a lot of piece of git that are way too foot-gunny for the general public; people could use it fine until they got stuck (what the fuck is a detached head?! does it have to do with the French Revolution?!)
On the other side there is "completionism", where FLOSS is "just a product" and its only goal is to always be better, faster, safer.
Correct, yes. FOSS are by and large functional works, not artistic ones. I think it isn't a good thing to put your personal social benefit over service to your users (or at least it needs a much more robust defense than saying "collectivism").
If social connections happen over FOSS, that's a nice bonus, but that's not why I've dedicated my life to it.
Correct, yes. FOSS are by and large functional works, not artistic ones
This is one of the things that has been pretty alarming to me watching both the pro-/anti-LLM rhetoric and the fascism witch-hunt rhetoric over the last few months. There seem to either be a large number of people, or a small number of very vocal people, who have tied their identities to some very specific things. Those things are shifting and they’re interpreting this as their identities being challenged or marginalized.
Being an Engineer is certainly part of my identity. So is being an uncle. So is being a neighbour. And being a son. And a good husband. And a good friend who will show up when someone’s having a hard time.
I’ve got somewhere on the order of 500 brew packages installed on my laptop and who knows how many across my various Linux boxes. I don’t know or care whether any of those were AI-assisted or not, or made by “fascists”, or whatever. I sometimes find bugs and if I can fix them I do and try to push those fixes upstream. Through and through, these are functional tools. I hang out on discord with friends who have no specific purpose other than to be friends. I’ve had a lot of clients over the years and have worked on a bunch of different products and with different teams. Some of those people are still in my life, and some of those projects were so cool that I still talk about them sometimes.
I can see the art and beauty in code. I’ve made code myself that I consider beautiful. I’ve also made hand-made slop that will run an experiment on a Friday afternoon and get deleted by Monday. I take pride in my work, but am not so attached to it that I find any of these new developments to be at all threatening to my identity as a person.
I don’t know or care whether any of those were AI-assisted or not, or made by “fascists”, or whatever.
It's a position of privilege to be able to ignore fascism because it doesn't affect you. Not everyone gets to just ignore it. Some of us do worry about whether we're supporting people who will in turn support causes that harm people we care about. That's not a witch hunt - witches aren't real.
You are correct that I am in a position of privilege, in a lot of ways. I have a very expensive laptop that I bought for myself that most people in the world couldn't afford. I live in a house that I "own" (technically the bank still owns a chunk of it) and I have no problem making my monthly mortgage payments or buying groceries. I own a vehicle and have no issue paying for fuel to get me places. There is almost no risk of violence in my life unless I go specifically seek it out.
Not everyone gets to just ignore it. Some of us do worry about whether we're supporting people who will in turn support causes that harm people we care about.
I'm not saying that I ignore these issues. I'm saying that I don't have a strong conviction that apt-get install or brew has any reasonable, significant nexus to people being actually harmed.
There are certainly people within my community that I support and oppose. There is harm being done in my community that I oppose and speak out against, and there is good being done in my community that I support both financially and spread the word about. There are a few non-profits that I consistently contribute my labour to, for free, to help them out when they need it. I also do these things because I'm in a position of privilege; I have the time and resources to be able to give these things freely without worrying too much. I don't feel guilty about that.
Actually, everyone does in fact get to ignore if code was written by fascists.
Saying "not everyone gets to ignore serial killers" is not a response to "I don't care if a serial killer wrote this package." The ways in which they don't get to ignore it are not usually related to the software the fascists wrote.
I don’t know or care whether any of those were AI-assisted or not, or made by “fascists”, or whatever.
Disagree with you here -- I do care about these things! I think AI-assisted in particular is very bimodal in practice -- either very shoddy or built to an exceptionally high standard. And I certainly don't want to use software helmed by fascists.
You're allowed to disagree! That's the beauty of it all! If those are things you want to worry about, you can. I've said this before on here but it seems the number has grown since last time:
% brew list -1 | wc -l
501
I have not and do not intend on auditing that list to figure out who the authors are. You're more than welcome to. If a piece of software sucks, I'll uninstall it.
Moreover, functional-first F/L OSS projects are where the common interest might force social connections to happen which would not happen otherwise. Sometimes, with good luck, it makes people on different sides of polarisation split to think of each other's views «well, it's all complicated», which needs way more luck with community-first communities…
Right, which is almost definitionally what pluralism is. What is the minimal subset of things we need to agree on to be able to work together? For me, that list is pretty small and mostly involves engineering philosophy along with a small set of moral/ethical philosophy that is largely, globally, uncontroversial. That’s not to say I don’t have more nuanced position on specific issues at global, national, provincial, and municipal level, just that agreeing with me on those specific issues is not a requirement for me to be able to work with you. I honestly don’t even know what my coworkers’ political positions are. It’s not something that comes up very often at all.
My take is that there are currently two ways of thinking about why FLOSS exists, or should exist. ... We can call one of these sides "collectivism". This frames FLOSS to be about accomplishing things together, enjoying the journey and bonds that big goals tend to create. The fun is the collective effort and challenge. ... "The journey is the destination", "FLOSS is the friends we made along the way", etc. On the other side there is "completionism", where FLOSS is "just a product" and its only goal is to always be better, faster, safer. ... Social bonds may happen, but colleagues are more coworkers than community. ... My assessment is that the collectivist framing finds FLOSS primarily a social exercise that rewards you with experiences, bonds, and ideas outside of your niche interests. Some times you even get good software as a bonus! The second framing sees FLOSS as a tool to scale the complexity
Almost ten years ago Allan Day described GNOME as Principled Software because of its commitment to always doing the right thing, ... This discussion is just another opportunity to continue to meet this expectation.
I lean far more towards the "second framing" and not this author's clearly preferred first framing, and I don't believe "LLM productivity is not real"... and yet I still dislike GenAI on principle, regarding its net effect on the world as bad, and refuse on principle to program with it (although I don't bug other people's projects not to), a stance that doesn't seem to fit into this article's view of the matter.
daveliepmann | 6 hours ago
This links to METR's early-2025 work. They've released a newer analysis from August 2025 that tells quite a different story. Note, that's still prior to the end-2025 models that changed a lot of devs' minds.
kornel | 5 hours ago
We've went through a few cycles of "ok, this model sucked but just wait for the next one", but in 2026 they actually started crossing the good-enough threshold.
It still depends on the type of the task, and code quality is meh, but there is a major difference between "hallucinated stackoverflow" of 2024 and "junior coder" of 2026.
rs86 | 3 hours ago
Caught me as well. Models have improved much after 3.5/3.7, and much of my productivity increase comes not from a single project but from being able to work on 4-5 projects simultaneously and maintain 10 by myself
wink | 8 hours ago
I kinda disagree with calling a proposal that is not an outright ban "LLM friendly".
Which doesn't mean I loved it, the linked one by one GNOME developer sounded better.
nemin | 7 hours ago
If I were an AI hypester, I'd be positively gleeful at every one of these overzealous judgements passed towards projects, who try to impose limits, but not enough to pass the purity line.
People, who haven't formed strong opinions yet will see the outrage, check what the actual policy was supposed to be, see that it was really not nearly as bad as it was portrayed as, and will thus put less weight behind arguments raised by antis.
It's like astroturfing, but it's your own very enemies doing all the work, while you don't have to do anything.
trenchant | 7 hours ago
Yep. I'm already checked out on most of the antis. They decry the loss of craft, their hobby and joy. And then follow up with no of course they haven't used it.
I wasn't around for the first "high-level languages" like FORTRAN, but I presume the guys who handwrote assembly and wired the memory themselves were just as salty about the coming onslaught of inefficiency and bad software.
dallen | 7 hours ago
I don't need to use something to pass judgement on it, especially when I can see its direct effects on my coworkers and peers. I'm getting really sick of this "you haven't even used it yourself" argument.
jfloren | 5 hours ago
To quote a modern philosopher, "you'll cowards don't even smoke crack"
elliotmorris | 3 hours ago
I worry about people who are heavily using LLMs who haven't overcome serious addiction before in their lives. They may not be able to recognize the feeling.
kolja | 7 hours ago
I kind of like it. It is an easy way for people to self-identify as "I didn't even take the time to try to understand the other side's argument.", making them easier to dismiss/ignore.
tonyarkles | 5 hours ago
On the flip side, I generally ask that question when the person is making an assertion that, in my personal experience using LLMs judiciously, seems either completely false or “was true 16 months ago but not now”.
wink | 7 hours ago
I don't really see how this is related. Having used it, not having changed my stance from being against it, I still think I could've judged it all the same just from observing the output.
It's a bit like reading bad fiction. Reading it is enough, I don't need to try writing it.
That said, I'm unenthusiastically ok with using it at work where it makes sense, but I prefer to keep my open source free from it. That might make me some capitalism apologist because I'd like my company to make money to pay me, but I don't see why most FLOSS development has to be be "improved" that way
mtset | 7 hours ago
Yeah, it was mostly real idiots like Edsger Dijkstra anyway. No need to listen to them. (/s, as I should hope is obvious.)
kaig | 6 hours ago
And they were right to do so, things have gotten worse and worse. If AI is so good then why does everything made with it suck so bad? I don't have to use AI to be subjected to its poisonous fruits against my will.
trenchant | 4 hours ago
I believe things had been getting worse already, especially since the advent of frameworks-of-the-week taking over the vast majority of webdev, and websites taking over the majority of human-computer-interaction. AI has simply accelerated the enshittification and the revealing of those without taste / fundamentals. Spotify was a broken POS before AI, and this post predates AI too. We're just accelerating the need for a reckoning with global software quality.
elliotmorris | 3 hours ago
Perhaps, but many are salty about much more than the coming onslaught of inefficiency and bad software. People dislike LLMs for reasons that are generally not normally acceptable to consider, sometimes even in egalitarian open-source environments. Often these reasons are illegible and hard to verbalize, and can cause people to have to confront uncomfortable ethical inconsistencies within themselves.
emk | 3 hours ago
This is a deep misunderstanding of the "completionist" project, or at least large parts of it.
The completionist project is profoundly political, dating right back to Stallman himself. The goal was to provide an operating system and a set of tools that preserved the freedom of end-users to do whatever they wanted with their computer. This is essentially an "anarchist" project, in a political sense: Devolve power to the people. Allow them to do anything they want with their hardware. And Stallman chose essentially anarchist means: Programmers would build GNU themselves, and give it away under a copyleft license, to prevent the commons from being enclosed.
Under the GNU Project's goals, nothing suggests that you couldn't just ask a benevolent Culture Mind to just implement the entire GNU system, document it extensively, and release it under copyleft. (Let's assume Culture Minds can legally apply copyleft to their own creations, based on the legal argument of "give the nice Rapid Offensive Units anything they want.") Boom, total end-user freedom, done in an afternoon.
A modest proposal: Since I'm mildly annoyed with people declaring that the completionists are apolitical, I'm going to propose an even more radical completionist project. To be clear: I do not necessarily endorse this project.
The goal: Every non-technical end user should not just have the hypothetical ability to modify their operating system and applications. Instead, we want to make this ability real, for every user. Minimum system requirements will be 32GB of GPU-like RAM and 500 GB/s of memory bandwidth, so this project will need to wait until after the next big RAM price crash. This system would include:
Now, the user not only has the full source code to their system, they also have a 100% local coding agent that they can talk to and ask to customize things. With luck, the guard rails and data protection will mostly hold up. Freedom can sometimes be messy, and this version might get messy indeed!
mtset | an hour ago
One not based on violating licenses or on vast amounts of RL data generated by underpaid and traumatized people in the global south, right?
... right?
tonyarkles | an hour ago
Honestly, reading this is giving me some serious nostalgia vibes. It’s not the same, at all, but I learned to program on a Commodore VIC-20 and most of the programs I started with came from books that I took out from the public library. Clearly not the same, but… spiritually I see it! The system was mutable. It didn’t take much study for me to be able to start understanding how the games from those books worked and then riff on them to make my own.
I’ll toss point 8. into the ring to see if you like it: a clever way for users to share their customizations with each other. I’d say git but there’s a lot of piece of git that are way too foot-gunny for the general public; people could use it fine until they got stuck (what the fuck is a detached head?! does it have to do with the French Revolution?!)
FeepingCreature | an hour ago
Anyway this sounds awesome. It's pretty close to the reason I run Gentoo, tbh.
sunshowers | 5 hours ago
Correct, yes. FOSS are by and large functional works, not artistic ones. I think it isn't a good thing to put your personal social benefit over service to your users (or at least it needs a much more robust defense than saying "collectivism").
If social connections happen over FOSS, that's a nice bonus, but that's not why I've dedicated my life to it.
tonyarkles | 4 hours ago
This is one of the things that has been pretty alarming to me watching both the pro-/anti-LLM rhetoric and the fascism witch-hunt rhetoric over the last few months. There seem to either be a large number of people, or a small number of very vocal people, who have tied their identities to some very specific things. Those things are shifting and they’re interpreting this as their identities being challenged or marginalized.
Being an Engineer is certainly part of my identity. So is being an uncle. So is being a neighbour. And being a son. And a good husband. And a good friend who will show up when someone’s having a hard time.
I’ve got somewhere on the order of 500 brew packages installed on my laptop and who knows how many across my various Linux boxes. I don’t know or care whether any of those were AI-assisted or not, or made by “fascists”, or whatever. I sometimes find bugs and if I can fix them I do and try to push those fixes upstream. Through and through, these are functional tools. I hang out on discord with friends who have no specific purpose other than to be friends. I’ve had a lot of clients over the years and have worked on a bunch of different products and with different teams. Some of those people are still in my life, and some of those projects were so cool that I still talk about them sometimes.
I can see the art and beauty in code. I’ve made code myself that I consider beautiful. I’ve also made hand-made slop that will run an experiment on a Friday afternoon and get deleted by Monday. I take pride in my work, but am not so attached to it that I find any of these new developments to be at all threatening to my identity as a person.
bmo | 4 hours ago
It's a position of privilege to be able to ignore fascism because it doesn't affect you. Not everyone gets to just ignore it. Some of us do worry about whether we're supporting people who will in turn support causes that harm people we care about. That's not a witch hunt - witches aren't real.
tonyarkles | 3 hours ago
You are correct that I am in a position of privilege, in a lot of ways. I have a very expensive laptop that I bought for myself that most people in the world couldn't afford. I live in a house that I "own" (technically the bank still owns a chunk of it) and I have no problem making my monthly mortgage payments or buying groceries. I own a vehicle and have no issue paying for fuel to get me places. There is almost no risk of violence in my life unless I go specifically seek it out.
I'm not saying that I ignore these issues. I'm saying that I don't have a strong conviction that
apt-get installorbrewhas any reasonable, significant nexus to people being actually harmed.There are certainly people within my community that I support and oppose. There is harm being done in my community that I oppose and speak out against, and there is good being done in my community that I support both financially and spread the word about. There are a few non-profits that I consistently contribute my labour to, for free, to help them out when they need it. I also do these things because I'm in a position of privilege; I have the time and resources to be able to give these things freely without worrying too much. I don't feel guilty about that.
FeepingCreature | an hour ago
Actually, everyone does in fact get to ignore if code was written by fascists.
Saying "not everyone gets to ignore serial killers" is not a response to "I don't care if a serial killer wrote this package." The ways in which they don't get to ignore it are not usually related to the software the fascists wrote.
sunshowers | 3 hours ago
Disagree with you here -- I do care about these things! I think AI-assisted in particular is very bimodal in practice -- either very shoddy or built to an exceptionally high standard. And I certainly don't want to use software helmed by fascists.
tonyarkles | 3 hours ago
You're allowed to disagree! That's the beauty of it all! If those are things you want to worry about, you can. I've said this before on here but it seems the number has grown since last time:
I have not and do not intend on auditing that list to figure out who the authors are. You're more than welcome to. If a piece of software sucks, I'll uninstall it.
k749gtnc9l3w | 4 hours ago
Moreover, functional-first F/L OSS projects are where the common interest might force social connections to happen which would not happen otherwise. Sometimes, with good luck, it makes people on different sides of polarisation split to think of each other's views «well, it's all complicated», which needs way more luck with community-first communities…
tonyarkles | 4 hours ago
Right, which is almost definitionally what pluralism is. What is the minimal subset of things we need to agree on to be able to work together? For me, that list is pretty small and mostly involves engineering philosophy along with a small set of moral/ethical philosophy that is largely, globally, uncontroversial. That’s not to say I don’t have more nuanced position on specific issues at global, national, provincial, and municipal level, just that agreeing with me on those specific issues is not a requirement for me to be able to work with you. I honestly don’t even know what my coworkers’ political positions are. It’s not something that comes up very often at all.
5d22b | 5 hours ago
I lean far more towards the "second framing" and not this author's clearly preferred first framing, and I don't believe "LLM productivity is not real"... and yet I still dislike GenAI on principle, regarding its net effect on the world as bad, and refuse on principle to program with it (although I don't bug other people's projects not to), a stance that doesn't seem to fit into this article's view of the matter.