The not-so-quiet goal of the LLM push by big tech was to homogenise intellectual labour such that it can be made fungible. When labour is fungible you can swap it out whenever you like for an alternative, decreasing the bargaining power of workers.
This is the most important reason that managers want "AI". It's the same reason they wanted offshoring to work circa 2005. It's the same reason they push "nobody is irreplaceable". The real question is why software developers go along with using and installing "AI".
Years of a very strong bargaining position and high wages leads to poor class consciousness, that only becomes clear when the balance shifts. We're a couple years into that at this point.
Lately our meetups have half saying they're part of the laid off club, while the rest grieve their state of employ. I see them commiserate together (that's not all we do, the bulk is still nerding out about programming.)
Point is I'm kind of astonished. My latest memo would've been quite taboo just two years ago.
The real question is why software developers go along with using and installing "AI".
I think that one's quite simple. Software developers are more like "ordinary people" than we like to think. Most ordinary people just do what they're told. It's no surprise that that applies to software developers too.
Then among the rest there's a significant portion who think AI tools do help them.
(I'm not intending to pass judgement on any of these groups by stating this.)
Tech Workers Coalition has been saying this since forever. AI is not a technology; it's a social project to destroy cognitive labor power.
The real question is why software developers go along with using and installing "AI".
they go along to keep their job, for fear of lagging behind in the rat race, or because they do not understand what's their individual and collective interest. If workers always acted in their collective self-interest, the world would be very different.
If everyone refused to use productivity improving technology then we’d all be a lot poorer. The answer to the “rich people get most of the benefits” problem is a recurring problem of how society is organized and how the majority of people are not organized except by the powerful.
the solution is indeed not to refuse productivity improvements, but allocate the surplus generate by that improvement to the benefit of workers and society, rather than further accumulation by the few. This economic model would collapse if that happened, and therefore must be replaced. Programmers, in such a scenario, wouldn't be destroyers of good life, but creators of wellbeing.
I heartily agree with most of the points made in this post, but I think it's unwise to place a lack of efficacy as the foundation of your objection to LLMs. There are many applications where buggy, crude, unmaintainable piles of plagiarized slop can be widely viewed as acceptable (mockups, one-off scripts) or even valuable because they don't work properly but offer a "moral crumple zone" by redirecting blame from human decision-makers (insurance claim processing, resume filtering). LLMs are inherently unreliable, but if you're willing to burn enough tokens and take enough pulls at the slot machine (or cloak the LLM in enough layers of automated slot-machine pulling), a broken clock can be correct several times a day, for a price. Efficacy can- to an extent- be brute-forced, and that's why LLMs are such a dangerous temptation.
I think it's better to keep the ethical objections first. Using slop is fundamentally irresponsible and harmful to our society and information environment because none of the LLMs that are available today are constructed without plagiarism at a massive scale- not just copyright infringement, but more broadly the total erasure of attribution and authorship. Extruding prose-shaped slop infinitely faster and more cheaply than it can be read and understood by humans undermines the social contracts of written language and produces a chilling effect on all genuine communication. Deferring to opaque boxes of matrix arithmetic is a total surrender of the essence of programming as a process of understanding and codifying precisely how to do things into merely "getting outputs you want, sometimes, somehow." I could go on.
These objections will not change if the LLM industry spends another hundred trillion dollars to distill every recorded word in human history into their models, or to bake present-day frontier models into ubiquitous ten-cent chips that run on a coin cell. Boiling the oceans and shredding all our libraries might let an LLM get the right answer, but it will not make them right.
You can't convince a corporation with ethics. But if LLMs are straight up not worth the money, that's gonna be what gets the c-suites to ditch them. And it seems that at the moment the customers are neither paying the full inference cost, nor are they seeing the promised productivity benifit.
I don't disagree with any of this fundamentally, with the caveat that to influence people you do have to speak their language. In the common scenario where a company wants to lean in on AI, they have already decided they aren't bothered by legal or ethical concerns. You can try to poke at that but you're probably unlikely to succeed. But challenging whether a specific approach makes sense, or whether using it for this particular project makes sense, might be arguments you're able to win.
I think it is still worth it to talk about ethics, because that still can influence the collective bargaining, and union demands would be as hard to ignore as the lack of efficacy of LLM use.
I heartily agree with most of the points made in this post, but I think it's unwise to place a lack of efficacy as the foundation of your objection to LLMs.
Your entire comment is well-reasoned and well-written. I think it's useful to observe that the (likely) main reason many of us tend to lead with an objection about the poor quality of the output is because that was the most striking thing when the hype machine spun up. Art, writing and programming output from the machinery built around these LLMs was so laughably bad that many of us jumped to point it out. And still do.
"It would be bad to use this slop in production" seemed like a very strong objection. You're likely correct that pushing ethical objections to the fore is stronger case. But I understand why it's more appealing to argue about first order badness (crappy output) than the second or third order consequences that you first need to educate people about before you can make your ethical case.
Art, writing and programming output from the machinery built around these LLMs was so laughably bad that many of us jumped to point it out. And still do.
I think it's a mistake to argue this way, unless you're exceptionally clear about what "laughably bad" means and you believe your audience agrees with or understands you. LLM outputs nowadays were the stuff of science fiction a decade ago, no matter how bad we may agree they are. Even Google in its heyday before enshittification could not answer my query with bespoke code that compiles --- or an image of my design that resembles it in the slightest. Even in the circumstances where you are right to say the code/image sucks, from the lay perspective it can come across as griping about the lack of cup holders on the Model T.
I'm less skeptical myself, however many years in we are, that LLMs can't be used for my work. My skepticism remains in whether they should be. I'm waiting to see what happens when the bill comes due for the wonder machine myself, and I'm hoping that the rest of us aren't expected to pay for it if it's bad.
A better way to look at it is to view the process as a genetic algorithm. It's not that you're pulling a lever in a slot machine and getting a random output each time. Agentic harnesses are effective because you have an evolutionary process. The model outputs something that's roughly correct, then it gets tested, model gets feedback and iterates. Through this process it converges on a solution that fits the parameters being tested. And that's not actually all that different from how humans write code either. You almost never solve a non-trivial problem in one shot. You write your first approximation and then iterate on it. The difference is that the LLM can do this process a lot faster.
I fully agree regarding the accountability argument however. LLM is a tool a human uses, and the user should be the one accountable for what they do with this tool.
We are 4 years into this ‘revolution’ and the industry has almost nothing to show for it.
This argument is a little "no true Scotsman". There are things to show, but I'm sure the author has heard of them and dismissed each for a different reason.
The avalanche of security vulnerabilities being found in existing major OSS projects
The AI rewrite of Bun from Zig to Rust
AI code review intercepting problems so humans don't have to
2 was a marketing stunt, it's not meaningful in the least.
3 is too fuzzy; the author already mentioned that LLM-directed review seems to produce garbage when it comes to his projects, so they've already brought that particular case up. This leaves 1, which I think is a fair argument. Not good enough to excuse several entire industries completely falling apart while "AI" has trillions put into it, but it's a fair argument for something.
Yes agents will produce trivialities on review but they will often find bugs that are hard to find except on the slowest review, and even more so the more you tell them to use various testing techniques.
LLM-directed review seems to produce garbage when it comes to his projects
There is a very weird effect where people get massively different tiers of performance out of AIs.
I use a Satisfactory mod (Ficsit Networks) that has yet to be updated to the current major version of the game. There's a fork somebody is running where he just has Codex read the issues every few hours. (Codex also wrote the port.) I have not even talked to the guy in days. Every issue and crash that comes up, we just slap it into Codex and have a fix the next day. There's issues where the bot is stumped (there's a weird rendering lag issue with the code ui for instance) but it's otherwise batting nine out of ten so far. As far as I can tell, it's just comfortable with this codebase and doesn't have any difficulty making changes. And (to my awareness) it's not being prompted, it's not being managed or guided. It just reads issues, responds, and makes releases with fixes, autonomously.
At the same time, people report that they can't get the LLMs to do anything in their codebase without causing further issues. I believe their experience, but something seriously strange is going on here, right? It's like we're bifurcating into people who have the magic aura that makes AIs work for them and people who don't.
It is very odd and I’m disappointed that we haven’t seen any studies on it yet, possibly because the big labs don’t want to admit that people who don’t get value from AI are out there.
My hunch is that people who feel that way are trying to one shot things that are obvious to them without the agent having access to the right tooling and context to figure out if what it’s done is correct. Possibly in codebases with minimal indicators of intent. But this really is just my hunch.
I took it as nothing "positive" or "groundbreaking" show for it, as I have had the same feeling. It has only introduced more problems, or at least its the problems that have become more prominent over advancements in the products being built.
I think that's the thing. Aside from the myriad of "tools for developers" being built on VC money in the hope that something will stick, most of the startup products, and a lot of the enterprise, are just as sloppy as before. Perhaps a little less formatting issues since LLMs are good at that, and more architecture problems that never get realized because these types of products never reach the scale where the cost is obvious and visible.
We just have all of these products faster now, and the problems of having so much slop, that we now call AI slop, is making the problems more prominent.
People fail to realize the dichotomy here. It's not
a) work getting done with AI
b) work getting done without AI
It's actually a trichotomy:
a) work getting done with AI
b) work getting done without AI
c) nothing getting done
Sure, some of b is going into a as AI gets better, but mostly it's c => a. A lot of that is "slop" that deservedly goes away soon after its creation, but not all of it.
As an example of c=>a, i.e. something that probably wouldn't have existed without vibecoding: Qobuz multiplatform client https://github.com/vicrodh/qbz -- sure, it's not making anyone any money (except perhaps to Qobuz itself in a small way), but it sure provides a lot of value. Yogthos's Jolt seems to be another one from this thread. I'm sure there are many others all over the place, many not so public.
Please tell me why this is being blamed on the tool instead of the person, again?
Who or what is responsible, again? This is literally the first thing about this that people need to understand and the open-source projects you refer to are fucking it up from the get-go. And yes, that includes my fave language of late, Zig. Because as it turns out, no one person understands everything, and Andrew is simply wrong on this.
IT IS ALWAYS THE PERSON WHO IS RESPONSIBLE. If someone's PR makes more work for you than not, give them 1 (final) warning and then if they do it again, BLOCK THEM. If their work contribution is actually fine, but you know it was done with LLM assistance and then you block them due just to that reason, THEN YOU ARE THE PROBLEM, NOT THE AI, BECAUSE YOU ARE IN A CULT. (A system of belief or bias that is not empirical.)
If this means establishing some sort of trust mechanism (which GitHub doesn't make easy, but could), then that is vastly superior to just outright "banning due to possible LLM assistance". Because 1) that assistance is rapidly heading to a point where it will be indispensable, which will make this nearsighted effort look really fucking stupid, and 2) the people who learn to work WITH LLM's instead of against them are going to absolutely end up having superior work output quantity AND quality.
Honestly, I welcome it. The more people reject LLM assistance, the wider the market is for people like me who are doing the complete opposite. DO IT! >..< The line is drawn and the die is cast! Let us see how it plays out.
Please tell me why this is being blamed on the tool instead of the person, again?
because the tool is what is allowing people that don't know about software development to create the PRs and push them posing as professional software engineers.
So, like I’ve been saying, this is the fault of those people, then, and not the tool. If people oppose this, GitHub should’ve created some kind of reputation or trust system by now, and they’ve done essentially nothing. Which is Github’s fault, not the tool’s.
this is like saying the kevlar armor's manufacturers are responsible for wars, not arm manufacturers. Also in this specific example, GitHub is both. Crazy logic.
Honestly you're coming across as unhinged here so I'm not sure if there's any discussion to be had. Can we dial it back?
Please tell me why this is being blamed on the tool instead of the person, again?
Because the problem very clearly got much worse when LLMs were introduced, and they lower the bar to make the bad contributions. A project has to deal with the reality they face. If they now get 100x the PRs and 99% are slop that wastes their time, they have to enact policy to keep the project sustainable, even if there are 1% of responsible users.
Honestly, I welcome it. The more people reject LLM assistance, the wider the market is for people like me who are doing the complete opposite.
I think this is what skeptics including the OP have been gesturing at, too. If LLMs are such a competitive advantage, it should become apparent pretty quickly. It has not yet. There isn't a sudden wave of high quality vibe coded projects that people prefer. If you think you can outcompete anyone, demonstrate it.
Your own biases are pretty clear from your post history here and on HN, you are obviously all-in; there can be almost no sensible discussion at that point.
That's ridiculous; I am actually welcome to all sensible discussion; the problem is that I encounter not just skeptics but ignorants who are both skeptical AND who have no remotely significant experience with it AND who think their opinion somehow still counts. Can you at least see how that is frustrating?
You would be right if this was an unfounded belief; it's not. I've hand-coded for 25+ years and agent-coded for about a year now. I have an informed opinion, by definition. (And on Twitter and other platforms, I am OFTEN bitching about the LLM-lying, etc. I just retweeted this! https://x.com/gergelyorosz/status/2088567020623151475?s=46&t=qWag1P6krA-5zkoLCeptNw )
This reminds me of the old "Mac vs. PC" arguments where 99% of the Mac people had used both Macs and PCs extensively and still preferred Macs, but 99% of the PC people had only used PC's (or, sorry, "that one time I used a Mac and couldn't find the Start menu" types), and the latter somehow believed their opinion was informed and thus mattered. You can guess which side I was on, on that debate, as well; it fits perfectly.
(aside: I HATE that there are managers essentially forcing devs to use AI. Absolutely the 100% wrong way to go about that, it’s like the Nazis controlling France and assuming it will just stick and that there won’t be a revolt…)
That's ridiculous; I am actually welcome to all sensible discussion; the problem is that I encounter not just skeptics but ignorants who are both skeptical AND who have no remotely significant experience with it AND who think their opinion somehow still counts. Can you at least see how that is frustrating?
Can you see that you sound like someone under a heavy AI psychosis to lots of knowledgeable experienced people - that you are calling skeptics and ignorants?
i’m also a knowledgeable experienced person, unless you are literally claiming that experiencing firsthand the thing you’re discussing, to be the opposite of “experience”, which would be a fantastical gaslighting attempt
To be fair, it's hard for anyone not using LLMs to trust anything that someone using LLMs says about using LLMs, because those people not using LLMs have seen the garbage slop that people using LLMs have dumped on them to read / accept / review / clean up.
I'm on both sides of the fence, and it's a painful position to be in, as you can imagine if you can visualize that idiom. LLMs do some things amazingly well, and I do use LLMs for those things. And I try to use them more and more for those things, without losing my ability to figure out those things myself. But I also get patches that were obviously made by LLMs, and where the person submitting the patches clearly has no idea what the LLM did and why it worked to pass the tests and why there's no way in Hades that I can accept that pile of slop that will inevitably lead to the collapse of human society as we know it.
I'm still firmly in the camp of: Never let the LLM spit out anything that you can't fully understand, and furthermore, make sure that you do fully understand it, even if that increases your own time cost by 10x or whatever. If the LLM output is good, and your total time cost represents a time savings, then that may be a good use case for an LLM.
What I genuinely don’t understand is how articles like this still have such widespread support in 2026 despite all the evidence of tremendous change in our industry. And this change is coming from everywhere.
Because most people do not attribute the changes to the intrinsic quality of technologies and approaches labeled as AI. There's a technical discussion, and a social discussion, for example about the war waged against tech workers by American VC funds to cut salaries. The changes are real, but a lot of people can believe that and at the same time believe that software can't reliably be produced with AI assistance.
100% agree on the problems with using it strictly for code generation; as well as the lack of good arguments and proofs that it helps with creating non-trivial software.
But to be honest, I find LLMs to be a massive boost when used as a thinking partner/research assistant. I would say that in many contexts, my productivity is at least 2x if not 3x or 4x thanks for this additional, enhanced cognitive ability they provide me with ;) Of course you need to understand and validate their output, but man, it helps so much; it feels like having a smart assistant available all the time; never tired and always eager to help
poor-quality demoware that no reputable organisation would touch with a barge pole
I cannot avoid remembering a related quote:
My job was to lead a team of experts in building something that reduced the degree to which Cliff had lied by just enough that, when the person who signed the check pressed the Power button, we wouldn't all be sent to jail.
(Snowden, about defense contracts)
So, maybe no reputable organisation, but some well-funded ones…
Assuming, for a moment, that we could set aside the context in which these tools exist and have been developed, and consider only the tool itself, I am very curious about the ethics of a certain style of use.
I have been trying to decide whether and how to fit LLMs into my work. I recently put some thoughts together on a very constrained approach to the use of LLMs, following a set of very broadly stated principles:
Prompts should be unique. (If the same prompt appears in the same form more than once, this is proof both that the knowledge it provides is valuable and also that the human has failed to develop it themselves.)
Prompting should reduce over time. (That is, at any given granularity, prompt volume should monotonically decrease over time. If the volume stays constant or grows, that means that a given tier of knowledge—e.g., placed somewhere on a roughly drawn concrete→abstract spectrum—is not only useful but also that the human has failed to develop their own reasoning capabilities. Of course, if the granularity correspondingly increases, total volume of use may be steady or increasing.)
Prompts should increase human knowledge. (The purpose of the prompt is to make the human more capable.)
The last point hints at it, I think, but the idea is constrain use to only that which improves human capability. In other words, the tool must be a tool for human growth.
For example, I recently read some news discussing challenges schools in the United States are facing in developing early childhood literacy. It seems that one ongoing debate involves the preferred methodology, and there are proponents of the “phonetics” approach and of the “whole word” approach. My curiosity was piqued by considering how these approaches might compare for early language learners of a language organised primarily around non-alphabetic writing systems.
It turns out that I can speak one such language to a certain degree, and I can read its non-alphabetic writing system (though I use a typing-used orientated around a non-Latin semi-syllabary.)
Of course, in English I can generally engage with academic research far outside of my own expertise. This isn't to claim that I need not acknowledge my own limitations, but rather than a general university education has prepared me to perform basic research tasks and to analyse and assess basic research results. So I feel fairly confident that I could, in fairly short order, dig around to find the textbooks and studies and research papers that form the basis of these two teaching approaches. It may take me a while to be able to fully understand the research and to truly draw my own opinions (and, to be honest, I strongly believe that if you're outside of a field, specifically if you don't have the ability to conduct your own counter-studies, you are very limited in what you can draw from a research study. This is why when I watch fitness videos on YouTube, the moment someone utter the word “ghrelin,” I close the tab.) In any case, it may take me a while, but I can get started today.
In the other language, I am at a total loss. I can read the newspaper, sure, but academic vocabulary in fields that outside of my direct experience is just too difficult. (I read an English-language book on a topic in ethnomusicology the other week and every time the author wrote the word “imbricate†,” I had to go for a walk around the block to calm my nerves. Just imagine if you weren't a native speaker…) Furthermore, anyone who has had to do basic research in a university undergraduate setting very broadly knows where to find and acquire research artefacts (and, for the English speaking world, they're usually online at some .edu or in some journal at a .org or maybe published via an FTP site on a .gov…)
So, in this particular circumstance, I feel completely unable to satisfy my curiosity. I utterly lack the skill to do basic research on this topic, and I don't really even know how to go about developing that skill, other than, say, going back to college… in another language… in another country. (Also, if you want to spend a couple of hours indulging your curiosity on a topic like this, you need very patient friends, family, neighbours, or work colleagues…)
So I spent some time asking an LLM. I am, in general, wholly disinterested in entertaining its outputs that are related to matters of judgment. I don't really even care that much for its summaries. But it's so much better at searching than I am (especially across a terminology barrier!) that it's just ridiculous. I can spend an hour roughly translating terms like “phonological awareness” or “orthographic approaches,” and I can corroborate my translations with source language materials, and yet still not find anything in my searches or miss whole swathes of documents by not accounting for common variations.
Now, I don't care enough about this topic to go to school for it, but I care enough about it to spend a few hours reading about it and getting some background knowledge, especially if I develop other knowledge or capabilities along the way. In this particular case, I would at least like to be better at searching for other (often non-academic) topics in the non-English language, and I would at least like to expand my vocabulary a bit.
In this case, it seems not only possible but uniquely beneficial to rely on the support of an LLM, and it seems like there is a way to use the LLM to build human knowledge and self-sufficiency. The long-sequence of prompting is pretty easy to guess. Mostly, the form is just slightly more indirect: “find DOIs for research papers on …” → then I go read or skim the paper → “this paper mentions … but doesn't give a citation; find other papers with the same reference, and find the citation” → then I go review the results (or do a few refinement cycles until I get the desired result, or hit a dead-end that I'd have to confer with a human about) → “where else is this paper published or mentioned” → then I review and save these sources for later.
The last step is key: if I'm going to use the LLM, I want to use in a way which allows me to not have to keep using it. Thus, I want my prompts to be unique, otherwise it means I haven't learnt anything. I want it to support me in doing something, but develop the capability myself, so if I keep asking the LLM questions, they are always at (and pushing forward!) the frontier of my knowledge, and I want to eventually develop capability so that I can outperform the LLM.
(In fact, through this approach, I have, in some narrow areas, become faster than the LLM.)
This seems like a very tool-based used of the LLM, and I don't know that it comes in conflict with the complaints the author of this article brings up. What are its ethical ramifications?
† “It is, on the contrary, fundamentally relational, conferred on individuals by communities that recognize them as partners in ongoing conversations, as imbricated in common ‘webs of interlocution’ (Drews Lucas 2017,) and, potentially, as committed to collective political action.”—and that's the English version…
FWIW I designed the Kit programming language and built its compiler during my 12 week parental leave back in 2018. Not only was it within three months, I was completely sleep deprived the whole time (hence why I thought designing a programming language was a good idea.) It included a website with a WASM-powered playground, and an NES emulator implemented in the language itself.
I'm not sure there's a ton of value in these "compiler-measuring contests" though.
It's pretty easy to build a toy language in a few weeks. That's not in any way comparable to having feature parity with a mature existing language and being able to run a wide set of official libraries. It's pretty disingenuous for you try and equate these things.
And I get direct value from Jolt because it allows me to write native apps using Clojure without needing the JVM. So, try to at least having a bit of intellectual integrity when making your counterpoints.
I'm really not sure why yours is real and mine is a toy and I'd love it if you would enlighten me.
feature parity with a mature existing language
Kit has seamless interop with C. It parses C headers and can use C interchangeably, similar to Zig. It also worked with third party libraries. The NES emulator I linked was using SDL - and parsing and understanding all of its headers to do so. That even includes making constants implemented as C preprocessor defines fit into Kit's type system which was not trivial.
I'm glad you're getting value from your thing too but I don't think one or the other is obviously a toy here, and I don't appreciate being accused of dishonesty.
Edit: Maybe we can come up with a more objective way to measure which one is a "toy" and which one is "real?"
Github stars: Kit has 1k, Jolt has 233.
Upvotes and comments on their respective lobste.rs discussions: Kit sits at 35/32 (with a much smaller user base back then) vs. Jolt with 24/6
Any other suggestions? Or do you want to retract the point that no one can do something as amazing as what your bot did in 3 months?
I'm not talking about having interop here. I'm talking about rebuilding an existing language that's been evolving for around 20 years now with all its quirks and subtelties. Not only does Jolt match Clojure semantics, it also implements a large subset of Java semantics and API which existing JVM Clojure libraries use. All of this is done on a completely different runtime, and it performs comparably to the JVM in a lot of cases.
I'm not sure how number of stars or lobse.rs discussion is a measure of complexity in your mind. Not to mention that your language has been around for nearly a decade while not getting significantly more attention than Jolt. So, not sure that's the comparison you want to be making.
An obvious suggestion is to look at the amount of code and actual functionality implemented. The fiber implementation alone in Jolt is more complex than most of your toy. The fact that you keep doubling down on pretending that these things are in any way comparable is frankly incredible. So, no, there's nothing to retract here. You clearly haven't spent even a few minutes to look at what Jolt actually is and what it does.
If we want to measure "complexity" and especially lines of code, I'm quite sure your agent-produced code is going to win, hands down. Whether that's desirable is a different question.
Again, if you think you can write something comparable with less code go ahead and put your money where your mouth is. Also, feel free to look at the amount of code in the official Clojure implementation and the Java standard library. The complexity here is inherent in the problem being solved, and doesn't come from code being agent produced as you're trying to insinuate. Either you genuinely don't understand what you're debating here or you're intentionally trolling. Which is it?
There's a lot of goalpost moving here which doesn't make for a productive discussion, but it's a bit silly to issue a challenge like this - it only takes one counterexample. I think people are capable of a lot more than you imagine they are.
For what it's worth I think your compiler with interop with C is a relatively comparable project. Clojure and Java do certainly have their complexities after 20 years. But so does the C abi.
I don't feel like contributing to what is obviously an unproductive conversation. But I did want to ask, how did you motivate yourself to work on a project like kit? How did you avoid short/long term burnout?
It was a little circuitous. I'm really intrinsically motivated to make games, which is how I got started programming as a kid. Kit was an extreme case of yak shaving where I had been using Haxe which I enjoyed and contributed to, had some ideas on how my workflow could be even better which the Haxe team didn't feel fit into that language, and decided to experiment myself. Eventually I ran into enough pain points that I decided to remove language from my complexity budget and moved to C++ where things are more stable than either Haxe or Kit. Putting the effort into the language and compiler wasn't moving me toward my real goal of making games, but it was a great experience nonetheless, and I'm still using the engine I built during that time, now ported to C++.
There aren't any goal posts moving here. What I said originally was that I was able to make complex project in under three months, and I challenged people to try building something on the same scale by hand.
You sealioned into the thread and tried to make an equivalence with your toy compiler. Then when I pointed out that what I built is far more complex, you started saying that I'm moving some unspecified goalposts. It should be obvious to anybody with even a modicum of integrity that you have to compare apples to apples to have a meaningful discussion.
I've been doing software development for over two decades now, and I know there's no way I could write something like this by hand in that amount of time. In fact, there have been plenty of abandoned attempts to port Clojure to run on Scheme over the years, and I'll leave it as an exercise for the reader to figure out why nobody managed to complete this herculean task manually before.
This is the entire problem with making claims like this.
No one can verify if you're correct or not. You are an anonymous poster on the internet making a claim of valuable complexity that indicates a category distinction. If you are correct, and your project is in fact, significantly more complex, then the level of effort to verify that claim is also significantly higher, to the degree that no reasonable person is going to do it.
Therefore, it follows that you can't really claim complexity as any sort of argument when posting anonymously on a public forum without anything else, success, reputation, etc, to back it up. You may know for a fact that you are right, but no one else can.
Anybody can go to the #jolt channel on Clojurians slack and see what people are currently building with it and what their experience is like. I'm also not posting anonymously. I've been around Clojure community for over a decade now, and I have books published, I maintain a web framework, and have lots of open source libraries with literally millions of users.
So, don't pretend that this is some inscrutable thing that nobody can verify.
I'm not pretending that. I'm just trying to get across, and it's a difficult concept to convey so I will be inaccurate, that the sort of verification you're gesturing at is not necessary for someone who already sees why you're right, and is unlikely to be performed in this context by someone who cannot. Hence, I believe claims presented towards the latter group in this unequivocal way are more or less pointless.
You are right, I was wrong to call you anonymous. The concept I was gesturing at was effectively anonymous for the purpose of how folks are going to consume a claim like this.
Again, this isn't some abstract debate. We are talking about a concrete compiler implementation for a well known language. Seems like the problem isn't with demonstrating what I said factually, but that there is a group of people who just want to argue without spending the time to learn the facts. And that's entirely their problem as far as I'm concerned.
I've presented a claim that I can defend on factual basis. If people want to honestly debate this claim they have to spend the time to investigate the facts. As the saying goes, no investigation, no right to speak.
As the saying goes, no investigation, no right to speak.
Okay fair enough. I merely worry that this inevitably means that the camp that has the most power to publish in volume will inevitably be the only camp capable of performing speech, regardless of quality distribution, due to the asymmetric effort implied in the verification effort.
I don't really follow what you're even trying to argue here. We are discussing two compiler implementations here, and whether they're comparable in terms of the scale of the problem they solve. What does any of this have to do with publishing in volumes?
Seems to me that actually understanding the subject people are attempting to debate is the bare minimum requirement. I'm not sure how you propose having any intelligible discussion where the participants don't even bother having a minimal knowledge of the subject they're discussing.
I think where you’re losing your audience is that you’re calling this other person’s language a toy, then failing to substantiate why it’s a toy. No-one is doing Jolt down.
If you look at the repo the author linked, it's very clear that it is a small hobby language. It's precisely the kind of thing you could hope to write by hand in 12 weeks or so.
The entire compiler is under 13k loc of Haskell. The type checker looks like the single biggest subsystem in the whole thing at about 3,500 lines. The README is upfront about calling the whole thing pre-alpha with features not fully implemented.
Kit has no persistent data structures, no lazy sequences, no concurrency model, and its metaprogramming is a term rewriting system over the typed AST.
By contrast, Jolt's compiler front end alone in Clojure weighs in at almost exactly the size of Kit's entire compiler. And that's the smallest part of jolt most of which is written in Scheme with over 50k loc. Overall source is literally an order of magnitude that of Kit while Jolt's tests alone are bigger than Kit's whole project.
And it's not just about lines of code, the projects are genuinely in different categories in terms of what each has to implement. Kit is a statically typed systems language that lowers to C through one backend, a unifier, and a small prelude. Jolt is reimplementing the semantics of a mature dynamic language, persistent data structures, dynamic dispatch, a reader, a loader, a namespace system, an nrepl server, FFI, and it has to get all of the semantics right for existing Clojure programs to actually run on it. For perspective, Clojure proper is similarly large with clojure.lang runtime alone being around 42k lines. So, yes, Kit is a toy in comparison.
It's a sophisticated compiler for a hobby project that implements things like type inference and algebraic data types. But it's simply not in the same league as a language like Clojure. And I personally find it insulting that the author tried to make an equivalnce between the two, then continued to double down talking about absurd metrics like github stars as a measure of complexity.
I think it would have behoved you to say this all much earlier and to be less thin skinned. Yes Jolt is impressive but I think you might be overly emotionally invested in people perceiving how great it is.
I'm not emotionally invested in people perceiving how great it is at all. I'm simply offended that Kit author would make an obvious false equivalence with my project and then refuse to even acknowledge doing that.
What's most disappointing here is how the thread predictably devolved into people just piling on instead of even trying to have an honest discussion on the subject.
The original point I was making is that I was able to make a complex piece of software much faster than I could have done previously. I presented this as something for the skeptics to consider.
And the skeptics predictably started doing mental gymnastics along with making personal attacks and false equivalences. Nobody has so far shown anything even remotely comparable being done by hand on the same time scale.
All this shows to me that people have already made up their minds on the subject, and no amount of evidence will convince them of anything. People made the conclusion that LLM development doesn't work and now it's basically no different from a cult. Rational discussion on the subject is clearly not possible here.
I've presented a claim that I can defend on factual basis.
You can trivially defend achievements and capabilities of your product, you can easily defend its current complexity.
Good luck however defending essential complexity, without putting the burden of proof on the naysayers...
As the saying goes, no investigation, no right to speak.
...There you are. But that's not how it works. You can't change people's minds without presenting evidence, and the kind of evidence you have access to with respect to essential complexity is likely very hard to convey.
Anybody can download Jolt and run the benchmarks in the repo.
And this is the fundamental problem: You just suggested that someone else invest (possibly waste) their time to validate some giant black box of likely-slop code sprawl that you had a team of GPU-monkeys-with-keyboards spit out.
You're asking for the impossible, like asking someone to review all of the 30,000 minutes of video that is being uploaded to Youtube ... in just the last minute! So if you speed up the video by 3x, and work 24 hours a day with no sleep, you can only fall behind by 43,000,000+ minutes of video every day. This is not a winnable war.
So much of our industry, our relationships, and our society itself is based on trust. The careless use of LLMs has degraded that currency, badly. Perhaps you, for example, carefully instructed the LLM, and personally reviewed, understood, and tested each line of code. Your project isn't slop! It's pristine. It's perfect.
But it's hidden inside 43,000,000 minutes of slop.
No, the fundamental problem is that people want to have opinions on things they haven't spent time investigating. Nobody is asking anyone to write vapid comments on subjects they don't want to spend the time to understand.
Also, we're not talking about some random thing I linked to without any context. We're discussing a project I built. You are talking to the author. There is no reason for me to try to trick you into anything. This project also reflects on my reputation as a developer and I'm putting that on the line by showing it discussing it. The fact that you don't even have the basic decency to acknowledge this says volumes.
My original point was that I made something that solves a non-trivial problem as an example for people who are skeptical about LLMs. Instead of having a rational discussion about the project, this thread turned into personal attacks.
It's pretty clear that there's just an echo chamber of people claiming that LLMs don't work, and when they're presented with counter examples they just glibly dismiss them because apparently it's jut too onerous to actually engage with other developers in good faith for you lot.
There aren't any goal posts moving here. What I said originally was that I was able to make complex project in under three months, and I challenged people to try building something on the same scale by hand.
Genuinely, this shows your ignorance of programming as a field. There are many, many examples over the last sixty years of programmers doing the same, or more, in lesser time scales independently. Maybe you should ask your LLM about it.
As someone who has taken the time to learn about the history of programming, your pet project is not particularly impressive, and the fact that you have been very visibly bad-mouthing a programmer who accomplished similar without the use of LLMs reflects very poorly on you, as a programmer, as a professional, and as a person.
Genuinely, this shows your ignorance of programming as a field. Feel free to provide examples of people writing a project with the scope of Clojure in a couple of months. As someone who has taken the time to learn about the history of programming, and having done development for decades, I can spot a troll when I see one.
I tried a challenge the other way and it did not go well at all. The person who tried to LLM the code I failed to write basically didn't understand the problem and didn't even bother to do all that much. Feel free to try either issue I mentioned; it should be even easier a year later with an LLM, right?
Also, it took me two weeks to implement by hand, ANS Forth for the Motorola 6809 [1]. An LLM should be able to chunk that out in a day, right? Feel free to try it. Oh, and my ANS Forth system passes the ANS Forth test suite for the wordsets I did implement. Or is this too trivial a project?
[1] I did not implement BLOCK, FACILITY, FILE, FLOATING, MEMORY or XCHAR (or the extensions) as those were more OS-dependent, took too much memory, or don't make sense on an 8-bit system.
I don't really have any context for what the problem being solved there is. I don't see how I'd be able to use LLMs to solve something that's outside my domain of expertise. LLMs aren't magic, they let you work faster, but they don't do thinking for you.
An LLM on its own is not able able to do much of anything, it requires the user to have domain expertise to apply it effectively. The reason I can build a Clojure compiler using these tools is because I have nearly two decades of experience working with the language. I know how it works internally, I know what the end solution needs to look like, and I know pitfalls to avoid.
Trying to solve a problem I have no familiarity with would just be me throwing darts at the board. Maybe the LLM will produce the right solution and maybe it will now. I'm not equipped to evaluate that.
And your challenge is different how? To take it anyone would need domain knowledge, right? Anyone trying to replicate your results by hand would need it. So is it a valid challenge?
My challenge is to build something of comparable scale by hand in the amount of time. It doesn't have to be the same thing I built, but something with a comparable level of complexity in the domain you're comfortable with. Or even better, simply show me a prior example of somebody doing that by hand.
Imllementing an existing language is obviously the easier task? You have the reference materials, all the PLD decisions have already been made for you.
I encourage you to try actually doing a task on this scale sometime to see whether you find it easy or not. While having lots of reference materials is very helpful, actually doing the work is a very labor intensive process, and putting all these things together in a way that actually works is not easy in any sense. In fact, there have been numerous previous attempts to do this, such as clj-scheme that never got to the point of being generally usable. If this was easy to do as you suggest, we would've seen this done a long time ago.
Making your own compiler with an order of magnitude less code where you can control the scope and make all the decisions is a far easier task.
I have found over an immense amount of cases that people who don't know how to do something tend to overestimate how long they take to make as well as how complex they are. Vibecoders and LLM overusers tend to never actually learn how anything works, so they tend to stay in that zone.
As much as the progenitor of this thread appears to be missing some of the point, I have some doubt that this is the case here, they look to be credentialed in their field, AI usage aside.
We cant be sure of this though, because I at least lack domain knowledge, and the traditional markers of trust have been almost completely eroded by LLMs when it comes to communicating with an audience even slightly outside of any specific domain. This is the point. Claims like this suck now, there's no path in which they accomplish anything.
Must be incredibly frustrating to be on the other side of it as well, assuming good faith. (Which is also much harder to do, given the tooling and the tone.)
Leaving aside the “how clever a human you have to be” aspect, there’s a shovelware aspect to Jolt that indubitably would have been very difficult to do in three months along with everything else. A lot of a product being mature is having a bunch of stuff and features. The frontier of what size of standard library a new language or implementation can ship with has definitely shifted.
So, would you refer to Clojure itself as shovelware as well, or are you just admitting that I was able to get Jolt up to parity in a backhanded way here? The fact that LLMs make it possible to get a project to a usable state faster is sort of the whole point here. What I find particularly telling is that there hasn't been a single constructive criticism of Jolt in this whole thread. There are plenty of people slinging mud at me, denigrating me, attacking my character and my work. But not a single comment actually engaging with the quality of the project or the complexity of the problem it solves. That says a lot about the whole skeptic position if you ask me.
The number of blog posts celebrating LLMs or conversely explaining away LLMs are legion. The next batch of trained LLMs will now know what they need to know to defend or dismiss LLMs. Mission accomplished.
The philosophy exuded by AI boosters still defies the last half-century of theory about what software development even is.
It is near the bottom but this one is so often overlooked. There are skill plugins like superpowers that attempt to let these models/harnesses make use of good practices like TDD, DRY, etc and they actually do improve the quality output in a remarkable way (there are still plenty of sneaky micro hallucinations, but that is besides the point).
But, none of the frontier labs includes those practices themselves in their harnesses or model training to any significant degree as far as I can tell. They all seem to focus on the sort of quick idea to code type development that works for prototypes but falls apart for larger projects.
Most people I know who use AI successfully in their workflow have had to spend A LOT of time putting up a ton of skill scaffolding and other instructions before they got to that point. Some of that comes down to project specific idiosyncrasies but a majority from what I have seen is the same sort of stuff as the superpowers skill set, trying to get these things to follow basic fundamental software development practices.
Then there is the fact that writing the code is just one aspect of the entire process to begin with. Something I have written about before to the point that the quote I am about to include quotes a previous comment as well, going full matryoshka doll.
I agree with a lot of the article but I also do think it is being overly optimistic about rigor in software engineering. Specifically this line
Plenty would kill to get the same kind of automated testing we treat as a given.
I have worked in a multitude of organizations and automated testing is still in this day and age not a given. Specifically robust automated tests at various levels and not part of the process. Software engineers all too often see themselves as the only checkpoint that is really needed and organizations try to eliminate parts of the processes they see as roadblocks.
I have written a a comment about this before so rather than repeating myself entirely I am going to include it below. In addition to that we as a software industry seem to collective suffer from short term memory loss where we are simply refusing to actually codify learned best practices and insist on reinventing the wheel every few years. I can already hear people scuff and point out that a lot of things have standardized over time and a lot of practices are shared between companies. But in my experience what has been standardized is mostly a shared vocabulary on loosely defined things that have wildly different meanings between companies and testing. Terminology like agile should be familiar to anyone. But also when we are talking about "testing" in my experience people think about wildly different highly specific things, not the holistic testing approach as a whole (see my comment below). Most developers will think about unit tests, some might think about a slightly expanded scope with component/integration unit tests. In some companies that is all the testing that will be done before things are yeeted to production even.
Anyway, no we certainly aren't special. But, we aren't the same either even though we probably should be a lot more similar in my view.
It also annoyed me but for a different reason as you are saying. The point of doing a code review, in a healthy organization, should be that it seen as one of many processes that together are responsible for good quality process. Not just a singular gateway that is responsible for something going to production. In fact, in many cases it shouldn't even be the last gateway depending on the complexity of the landscape, integration with other applications, etc. But, unfortunately for many teams and companies the QA process after code is merged is something that might as well not exist anymore.
Anyway, I am a firm believer of everyone being responsible who is involved in the process. However not to diffuse responsibility but because quality should be a integral part of the process before code is even written. Basic boring stuff like three amigo sessions (or whatever method used to get various disciplines involved in talking about spec and requirements), test driven development methods, basic static code analysis tooling integrated in IDEs and various checkpoints, QA/Test automation expertise other than developers themselves integrated in teams and much more.
I know that for many people this is not a reality they ever encounter. But if we are making the comparison to civil engineering. The developer shouldn't be both the engineer and construction worker for the bridge in many cases. Even if they are, in civil engineering there also never is one person who is responsible as the practices I mentioned are very much part of the process there as well. An engineer needs to document everything, make all the necessary calculations for load bearing, have others re-check those calculations, then often get approvals from relevant government bodies who will double check various things. Then during construction there are also many more points where things are audited, checked, approved, etc. Even for simpler construction like that of a single family house this is still the case. Plans need to be drawn up, someone needs to sign of based on calculations, permits need to be given based on those plans and at various points in the process inspections need to be done.
Yes, there are people who have more responsibility than other but overall there never is a single person who has the only responsibility. In fact whenever there is a engineering disaster you often see that it has been a failure in the overall process where at multiple points things should have been stopped but were not.
We like to compare ourselves in software development with civil engineering. But if we do that we need to realize that our processes very often are just not on the same level at all. At least not as far as constructing bridges goes. In the best case scenario we often find ourselves at the level of housing developers who build the cheapest possible houses and try to work around a lot of the legally requirements.
bediger4000 | a day ago
This is the most important reason that managers want "AI". It's the same reason they wanted offshoring to work circa 2005. It's the same reason they push "nobody is irreplaceable". The real question is why software developers go along with using and installing "AI".
bmo | a day ago
Years of a very strong bargaining position and high wages leads to poor class consciousness, that only becomes clear when the balance shifts. We're a couple years into that at this point.
abnercoimbre | 17 hours ago
Lately our meetups have half saying they're part of the laid off club, while the rest grieve their state of employ. I see them commiserate together (that's not all we do, the bulk is still nerding out about programming.)
Point is I'm kind of astonished. My latest memo would've been quite taboo just two years ago.
elliotmorris | 7 hours ago
Well that's a scary memo.
abnercoimbre | 2 hours ago
You won't believe the failure mode American devs can fall into.
Yogthos | a day ago
That's literally why Java exists, and large companies have already ironed this process out a long time ago.
mftrhu | a day ago
Go is another language which comes to mind.
pmarreck | 16 hours ago
Yes. Literally created to create ephemeral crapware.
cpurdy | 4 hours ago
To be fair, the ephemeral crapware concept came first. Go was just responding to the demand for more of it, cheaper, faster.
gspr | 12 hours ago
I think that one's quite simple. Software developers are more like "ordinary people" than we like to think. Most ordinary people just do what they're told. It's no surprise that that applies to software developers too.
Then among the rest there's a significant portion who think AI tools do help them.
(I'm not intending to pass judgement on any of these groups by stating this.)
chobeat | 5 hours ago
Tech Workers Coalition has been saying this since forever. AI is not a technology; it's a social project to destroy cognitive labor power.
they go along to keep their job, for fear of lagging behind in the rat race, or because they do not understand what's their individual and collective interest. If workers always acted in their collective self-interest, the world would be very different.
Student | 6 hours ago
If everyone refused to use productivity improving technology then we’d all be a lot poorer. The answer to the “rich people get most of the benefits” problem is a recurring problem of how society is organized and how the majority of people are not organized except by the powerful.
chobeat | 5 hours ago
the solution is indeed not to refuse productivity improvements, but allocate the surplus generate by that improvement to the benefit of workers and society, rather than further accumulation by the few. This economic model would collapse if that happened, and therefore must be replaced. Programmers, in such a scenario, wouldn't be destroyers of good life, but creators of wellbeing.
chobeat | 5 hours ago
terretta | 39 minutes ago
It's the same reason they wanted Java to work circa 2000s.
Internet_Janitor | a day ago
I heartily agree with most of the points made in this post, but I think it's unwise to place a lack of efficacy as the foundation of your objection to LLMs. There are many applications where buggy, crude, unmaintainable piles of plagiarized slop can be widely viewed as acceptable (mockups, one-off scripts) or even valuable because they don't work properly but offer a "moral crumple zone" by redirecting blame from human decision-makers (insurance claim processing, resume filtering). LLMs are inherently unreliable, but if you're willing to burn enough tokens and take enough pulls at the slot machine (or cloak the LLM in enough layers of automated slot-machine pulling), a broken clock can be correct several times a day, for a price. Efficacy can- to an extent- be brute-forced, and that's why LLMs are such a dangerous temptation.
I think it's better to keep the ethical objections first. Using slop is fundamentally irresponsible and harmful to our society and information environment because none of the LLMs that are available today are constructed without plagiarism at a massive scale- not just copyright infringement, but more broadly the total erasure of attribution and authorship. Extruding prose-shaped slop infinitely faster and more cheaply than it can be read and understood by humans undermines the social contracts of written language and produces a chilling effect on all genuine communication. Deferring to opaque boxes of matrix arithmetic is a total surrender of the essence of programming as a process of understanding and codifying precisely how to do things into merely "getting outputs you want, sometimes, somehow." I could go on.
These objections will not change if the LLM industry spends another hundred trillion dollars to distill every recorded word in human history into their models, or to bake present-day frontier models into ubiquitous ten-cent chips that run on a coin cell. Boiling the oceans and shredding all our libraries might let an LLM get the right answer, but it will not make them right.
hjvt | a day ago
You can't convince a corporation with ethics. But if LLMs are straight up not worth the money, that's gonna be what gets the c-suites to ditch them. And it seems that at the moment the customers are neither paying the full inference cost, nor are they seeing the promised productivity benifit.
bmo | a day ago
I don't disagree with any of this fundamentally, with the caveat that to influence people you do have to speak their language. In the common scenario where a company wants to lean in on AI, they have already decided they aren't bothered by legal or ethical concerns. You can try to poke at that but you're probably unlikely to succeed. But challenging whether a specific approach makes sense, or whether using it for this particular project makes sense, might be arguments you're able to win.
hjvt | a day ago
I think it is still worth it to talk about ethics, because that still can influence the collective bargaining, and union demands would be as hard to ignore as the lack of efficacy of LLM use.
hoistbypetard | a day ago
Your entire comment is well-reasoned and well-written. I think it's useful to observe that the (likely) main reason many of us tend to lead with an objection about the poor quality of the output is because that was the most striking thing when the hype machine spun up. Art, writing and programming output from the machinery built around these LLMs was so laughably bad that many of us jumped to point it out. And still do.
"It would be bad to use this slop in production" seemed like a very strong objection. You're likely correct that pushing ethical objections to the fore is stronger case. But I understand why it's more appealing to argue about first order badness (crappy output) than the second or third order consequences that you first need to educate people about before you can make your ethical case.
cole-k | 4 hours ago
I think it's a mistake to argue this way, unless you're exceptionally clear about what "laughably bad" means and you believe your audience agrees with or understands you. LLM outputs nowadays were the stuff of science fiction a decade ago, no matter how bad we may agree they are. Even Google in its heyday before enshittification could not answer my query with bespoke code that compiles --- or an image of my design that resembles it in the slightest. Even in the circumstances where you are right to say the code/image sucks, from the lay perspective it can come across as griping about the lack of cup holders on the Model T.
I'm less skeptical myself, however many years in we are, that LLMs can't be used for my work. My skepticism remains in whether they should be. I'm waiting to see what happens when the bill comes due for the wonder machine myself, and I'm hoping that the rest of us aren't expected to pay for it if it's bad.
Yogthos | 23 hours ago
A better way to look at it is to view the process as a genetic algorithm. It's not that you're pulling a lever in a slot machine and getting a random output each time. Agentic harnesses are effective because you have an evolutionary process. The model outputs something that's roughly correct, then it gets tested, model gets feedback and iterates. Through this process it converges on a solution that fits the parameters being tested. And that's not actually all that different from how humans write code either. You almost never solve a non-trivial problem in one shot. You write your first approximation and then iterate on it. The difference is that the LLM can do this process a lot faster.
I fully agree regarding the accountability argument however. LLM is a tool a human uses, and the user should be the one accountable for what they do with this tool.
alexjurkiewicz | 17 hours ago
This argument is a little "no true Scotsman". There are things to show, but I'm sure the author has heard of them and dismissed each for a different reason.
gonz | 11 hours ago
2 was a marketing stunt, it's not meaningful in the least.
3 is too fuzzy; the author already mentioned that LLM-directed review seems to produce garbage when it comes to his projects, so they've already brought that particular case up. This leaves 1, which I think is a fair argument. Not good enough to excuse several entire industries completely falling apart while "AI" has trillions put into it, but it's a fair argument for something.
Student | 6 hours ago
Yes agents will produce trivialities on review but they will often find bugs that are hard to find except on the slowest review, and even more so the more you tell them to use various testing techniques.
FeepingCreature | 4 hours ago
There is a very weird effect where people get massively different tiers of performance out of AIs.
I use a Satisfactory mod (Ficsit Networks) that has yet to be updated to the current major version of the game. There's a fork somebody is running where he just has Codex read the issues every few hours. (Codex also wrote the port.) I have not even talked to the guy in days. Every issue and crash that comes up, we just slap it into Codex and have a fix the next day. There's issues where the bot is stumped (there's a weird rendering lag issue with the code ui for instance) but it's otherwise batting nine out of ten so far. As far as I can tell, it's just comfortable with this codebase and doesn't have any difficulty making changes. And (to my awareness) it's not being prompted, it's not being managed or guided. It just reads issues, responds, and makes releases with fixes, autonomously.
At the same time, people report that they can't get the LLMs to do anything in their codebase without causing further issues. I believe their experience, but something seriously strange is going on here, right? It's like we're bifurcating into people who have the magic aura that makes AIs work for them and people who don't.
Student | an hour ago
It is very odd and I’m disappointed that we haven’t seen any studies on it yet, possibly because the big labs don’t want to admit that people who don’t get value from AI are out there.
My hunch is that people who feel that way are trying to one shot things that are obvious to them without the agent having access to the right tooling and context to figure out if what it’s done is correct. Possibly in codebases with minimal indicators of intent. But this really is just my hunch.
alanmeira | 4 hours ago
What is it to show? Most software got worse, we have proof of that.
The new software that is successful. Aren't they AI tools to create AI tools?
2 even proves my point as it made Bun lose adoption from important projects, not gain it. So it made the software worse for most people.
Student | an hour ago
What is that proof?
orgnizedmess | 11 hours ago
I took it as nothing "positive" or "groundbreaking" show for it, as I have had the same feeling. It has only introduced more problems, or at least its the problems that have become more prominent over advancements in the products being built.
zladuric | 7 hours ago
I think that's the thing. Aside from the myriad of "tools for developers" being built on VC money in the hope that something will stick, most of the startup products, and a lot of the enterprise, are just as sloppy as before. Perhaps a little less formatting issues since LLMs are good at that, and more architecture problems that never get realized because these types of products never reach the scale where the cost is obvious and visible.
We just have all of these products faster now, and the problems of having so much slop, that we now call AI slop, is making the problems more prominent.
vegai | 13 hours ago
People fail to realize the dichotomy here. It's not
It's actually a trichotomy:
Sure, some of b is going into a as AI gets better, but mostly it's c => a. A lot of that is "slop" that deservedly goes away soon after its creation, but not all of it.
As an example of c=>a, i.e. something that probably wouldn't have existed without vibecoding: Qobuz multiplatform client https://github.com/vicrodh/qbz -- sure, it's not making anyone any money (except perhaps to Qobuz itself in a small way), but it sure provides a lot of value. Yogthos's Jolt seems to be another one from this thread. I'm sure there are many others all over the place, many not so public.
pmarreck | 16 hours ago
“The AI PRs I receive are rubbish” is severe selection bias.
bmo | 16 hours ago
Maybe, but it matches a broad trend in open source. How many projects are locking down or banning LLMs due to contribution quality?
pmarreck | 6 hours ago
Please tell me why this is being blamed on the tool instead of the person, again?
Who or what is responsible, again? This is literally the first thing about this that people need to understand and the open-source projects you refer to are fucking it up from the get-go. And yes, that includes my fave language of late, Zig. Because as it turns out, no one person understands everything, and Andrew is simply wrong on this.
IT IS ALWAYS THE PERSON WHO IS RESPONSIBLE. If someone's PR makes more work for you than not, give them 1 (final) warning and then if they do it again, BLOCK THEM. If their work contribution is actually fine, but you know it was done with LLM assistance and then you block them due just to that reason, THEN YOU ARE THE PROBLEM, NOT THE AI, BECAUSE YOU ARE IN A CULT. (A system of belief or bias that is not empirical.)
If this means establishing some sort of trust mechanism (which GitHub doesn't make easy, but could), then that is vastly superior to just outright "banning due to possible LLM assistance". Because 1) that assistance is rapidly heading to a point where it will be indispensable, which will make this nearsighted effort look really fucking stupid, and 2) the people who learn to work WITH LLM's instead of against them are going to absolutely end up having superior work output quantity AND quality.
Honestly, I welcome it. The more people reject LLM assistance, the wider the market is for people like me who are doing the complete opposite. DO IT! >..< The line is drawn and the die is cast! Let us see how it plays out.
alanmeira | 4 hours ago
because the tool is what is allowing people that don't know about software development to create the PRs and push them posing as professional software engineers.
vegai | 4 hours ago
People actually do that? Any idea why?
pmarreck | 2 hours ago
So, like I’ve been saying, this is the fault of those people, then, and not the tool. If people oppose this, GitHub should’ve created some kind of reputation or trust system by now, and they’ve done essentially nothing. Which is Github’s fault, not the tool’s.
mtset | 2 hours ago
Why can things be the fault of Github, who make a tool, but not AI companies and the tools they make?
chobeat | 2 hours ago
this is like saying the kevlar armor's manufacturers are responsible for wars, not arm manufacturers. Also in this specific example, GitHub is both. Crazy logic.
bmo | 4 hours ago
Honestly you're coming across as unhinged here so I'm not sure if there's any discussion to be had. Can we dial it back?
Because the problem very clearly got much worse when LLMs were introduced, and they lower the bar to make the bad contributions. A project has to deal with the reality they face. If they now get 100x the PRs and 99% are slop that wastes their time, they have to enact policy to keep the project sustainable, even if there are 1% of responsible users.
I think this is what skeptics including the OP have been gesturing at, too. If LLMs are such a competitive advantage, it should become apparent pretty quickly. It has not yet. There isn't a sudden wave of high quality vibe coded projects that people prefer. If you think you can outcompete anyone, demonstrate it.
gonz | 11 hours ago
Your own biases are pretty clear from your post history here and on HN, you are obviously all-in; there can be almost no sensible discussion at that point.
pmarreck | 6 hours ago
That's ridiculous; I am actually welcome to all sensible discussion; the problem is that I encounter not just skeptics but ignorants who are both skeptical AND who have no remotely significant experience with it AND who think their opinion somehow still counts. Can you at least see how that is frustrating?
You would be right if this was an unfounded belief; it's not. I've hand-coded for 25+ years and agent-coded for about a year now. I have an informed opinion, by definition. (And on Twitter and other platforms, I am OFTEN bitching about the LLM-lying, etc. I just retweeted this! https://x.com/gergelyorosz/status/2088567020623151475?s=46&t=qWag1P6krA-5zkoLCeptNw )
This reminds me of the old "Mac vs. PC" arguments where 99% of the Mac people had used both Macs and PCs extensively and still preferred Macs, but 99% of the PC people had only used PC's (or, sorry, "that one time I used a Mac and couldn't find the Start menu" types), and the latter somehow believed their opinion was informed and thus mattered. You can guess which side I was on, on that debate, as well; it fits perfectly.
(aside: I HATE that there are managers essentially forcing devs to use AI. Absolutely the 100% wrong way to go about that, it’s like the Nazis controlling France and assuming it will just stick and that there won’t be a revolt…)
Anyway, human DNA is vibecoded. >..<
alanmeira | 4 hours ago
Can you see that you sound like someone under a heavy AI psychosis to lots of knowledgeable experienced people - that you are calling skeptics and ignorants?
pmarreck | 2 hours ago
i’m also a knowledgeable experienced person, unless you are literally claiming that experiencing firsthand the thing you’re discussing, to be the opposite of “experience”, which would be a fantastical gaslighting attempt
peter-leonov | 6 hours ago
I do and it works uncannily well. Give it a try. Seriously.
cpurdy | 4 hours ago
To be fair, it's hard for anyone not using LLMs to trust anything that someone using LLMs says about using LLMs, because those people not using LLMs have seen the garbage slop that people using LLMs have dumped on them to read / accept / review / clean up.
I'm on both sides of the fence, and it's a painful position to be in, as you can imagine if you can visualize that idiom. LLMs do some things amazingly well, and I do use LLMs for those things. And I try to use them more and more for those things, without losing my ability to figure out those things myself. But I also get patches that were obviously made by LLMs, and where the person submitting the patches clearly has no idea what the LLM did and why it worked to pass the tests and why there's no way in Hades that I can accept that pile of slop that will inevitably lead to the collapse of human society as we know it.
I'm still firmly in the camp of: Never let the LLM spit out anything that you can't fully understand, and furthermore, make sure that you do fully understand it, even if that increases your own time cost by 10x or whatever. If the LLM output is good, and your total time cost represents a time savings, then that may be a good use case for an LLM.
mitsuhiko | 3 hours ago
What I genuinely don’t understand is how articles like this still have such widespread support in 2026 despite all the evidence of tremendous change in our industry. And this change is coming from everywhere.
altano | an hour ago
chobeat | an hour ago
Because most people do not attribute the changes to the intrinsic quality of technologies and approaches labeled as AI. There's a technical discussion, and a social discussion, for example about the war waged against tech workers by American VC funds to cut salaries. The changes are real, but a lot of people can believe that and at the same time believe that software can't reliably be produced with AI assistance.
BinaryIgor | 10 hours ago
100% agree on the problems with using it strictly for code generation; as well as the lack of good arguments and proofs that it helps with creating non-trivial software.
But to be honest, I find LLMs to be a massive boost when used as a thinking partner/research assistant. I would say that in many contexts, my productivity is at least 2x if not 3x or 4x thanks for this additional, enhanced cognitive ability they provide me with ;) Of course you need to understand and validate their output, but man, it helps so much; it feels like having a smart assistant available all the time; never tired and always eager to help
k749gtnc9l3w | 6 hours ago
Article:
I cannot avoid remembering a related quote:
(Snowden, about defense contracts)
So, maybe no reputable organisation, but some well-funded ones…
dutc | 22 hours ago
Assuming, for a moment, that we could set aside the context in which these tools exist and have been developed, and consider only the tool itself, I am very curious about the ethics of a certain style of use.
I have been trying to decide whether and how to fit LLMs into my work. I recently put some thoughts together on a very constrained approach to the use of LLMs, following a set of very broadly stated principles:
The last point hints at it, I think, but the idea is constrain use to only that which improves human capability. In other words, the tool must be a tool for human growth.
For example, I recently read some news discussing challenges schools in the United States are facing in developing early childhood literacy. It seems that one ongoing debate involves the preferred methodology, and there are proponents of the “phonetics” approach and of the “whole word” approach. My curiosity was piqued by considering how these approaches might compare for early language learners of a language organised primarily around non-alphabetic writing systems.
It turns out that I can speak one such language to a certain degree, and I can read its non-alphabetic writing system (though I use a typing-used orientated around a non-Latin semi-syllabary.)
Of course, in English I can generally engage with academic research far outside of my own expertise. This isn't to claim that I need not acknowledge my own limitations, but rather than a general university education has prepared me to perform basic research tasks and to analyse and assess basic research results. So I feel fairly confident that I could, in fairly short order, dig around to find the textbooks and studies and research papers that form the basis of these two teaching approaches. It may take me a while to be able to fully understand the research and to truly draw my own opinions (and, to be honest, I strongly believe that if you're outside of a field, specifically if you don't have the ability to conduct your own counter-studies, you are very limited in what you can draw from a research study. This is why when I watch fitness videos on YouTube, the moment someone utter the word “ghrelin,” I close the tab.) In any case, it may take me a while, but I can get started today.
In the other language, I am at a total loss. I can read the newspaper, sure, but academic vocabulary in fields that outside of my direct experience is just too difficult. (I read an English-language book on a topic in ethnomusicology the other week and every time the author wrote the word “imbricate†,” I had to go for a walk around the block to calm my nerves. Just imagine if you weren't a native speaker…) Furthermore, anyone who has had to do basic research in a university undergraduate setting very broadly knows where to find and acquire research artefacts (and, for the English speaking world, they're usually online at some .edu or in some journal at a .org or maybe published via an FTP site on a .gov…)
So, in this particular circumstance, I feel completely unable to satisfy my curiosity. I utterly lack the skill to do basic research on this topic, and I don't really even know how to go about developing that skill, other than, say, going back to college… in another language… in another country. (Also, if you want to spend a couple of hours indulging your curiosity on a topic like this, you need very patient friends, family, neighbours, or work colleagues…)
So I spent some time asking an LLM. I am, in general, wholly disinterested in entertaining its outputs that are related to matters of judgment. I don't really even care that much for its summaries. But it's so much better at searching than I am (especially across a terminology barrier!) that it's just ridiculous. I can spend an hour roughly translating terms like “phonological awareness” or “orthographic approaches,” and I can corroborate my translations with source language materials, and yet still not find anything in my searches or miss whole swathes of documents by not accounting for common variations.
Now, I don't care enough about this topic to go to school for it, but I care enough about it to spend a few hours reading about it and getting some background knowledge, especially if I develop other knowledge or capabilities along the way. In this particular case, I would at least like to be better at searching for other (often non-academic) topics in the non-English language, and I would at least like to expand my vocabulary a bit.
In this case, it seems not only possible but uniquely beneficial to rely on the support of an LLM, and it seems like there is a way to use the LLM to build human knowledge and self-sufficiency. The long-sequence of prompting is pretty easy to guess. Mostly, the form is just slightly more indirect: “find DOIs for research papers on …” → then I go read or skim the paper → “this paper mentions … but doesn't give a citation; find other papers with the same reference, and find the citation” → then I go review the results (or do a few refinement cycles until I get the desired result, or hit a dead-end that I'd have to confer with a human about) → “where else is this paper published or mentioned” → then I review and save these sources for later.
The last step is key: if I'm going to use the LLM, I want to use in a way which allows me to not have to keep using it. Thus, I want my prompts to be unique, otherwise it means I haven't learnt anything. I want it to support me in doing something, but develop the capability myself, so if I keep asking the LLM questions, they are always at (and pushing forward!) the frontier of my knowledge, and I want to eventually develop capability so that I can outperform the LLM.
(In fact, through this approach, I have, in some narrow areas, become faster than the LLM.)
This seems like a very tool-based used of the LLM, and I don't know that it comes in conflict with the complaints the author of this article brings up. What are its ethical ramifications?
† “It is, on the contrary, fundamentally relational, conferred on individuals by communities that recognize them as partners in ongoing conversations, as imbricated in common ‘webs of interlocution’ (Drews Lucas 2017,) and, potentially, as committed to collective political action.”—and that's the English version…
Yogthos | 23 hours ago
I welcome any LLM skeptic to build something equivalent to Jolt in three months.
bmo | 23 hours ago
FWIW I designed the Kit programming language and built its compiler during my 12 week parental leave back in 2018. Not only was it within three months, I was completely sleep deprived the whole time (hence why I thought designing a programming language was a good idea.) It included a website with a WASM-powered playground, and an NES emulator implemented in the language itself.
I'm not sure there's a ton of value in these "compiler-measuring contests" though.
mk12 | 22 hours ago
You may want to update the website link on the repo (kitlang.org), it's sending me to a gambling site.
bmo | 22 hours ago
Thanks, the project is dead at this point and someone snatched the domain. Removed that link.
Yogthos | 23 hours ago
It's pretty easy to build a toy language in a few weeks. That's not in any way comparable to having feature parity with a mature existing language and being able to run a wide set of official libraries. It's pretty disingenuous for you try and equate these things.
And I get direct value from Jolt because it allows me to write native apps using Clojure without needing the JVM. So, try to at least having a bit of intellectual integrity when making your counterpoints.
bmo | 23 hours ago
I'm really not sure why yours is real and mine is a toy and I'd love it if you would enlighten me.
Kit has seamless interop with C. It parses C headers and can use C interchangeably, similar to Zig. It also worked with third party libraries. The NES emulator I linked was using SDL - and parsing and understanding all of its headers to do so. That even includes making constants implemented as C preprocessor defines fit into Kit's type system which was not trivial.
I'm glad you're getting value from your thing too but I don't think one or the other is obviously a toy here, and I don't appreciate being accused of dishonesty.
Edit: Maybe we can come up with a more objective way to measure which one is a "toy" and which one is "real?"
Any other suggestions? Or do you want to retract the point that no one can do something as amazing as what your bot did in 3 months?
Yogthos | 22 hours ago
I'm not talking about having interop here. I'm talking about rebuilding an existing language that's been evolving for around 20 years now with all its quirks and subtelties. Not only does Jolt match Clojure semantics, it also implements a large subset of Java semantics and API which existing JVM Clojure libraries use. All of this is done on a completely different runtime, and it performs comparably to the JVM in a lot of cases.
I'm not sure how number of stars or lobse.rs discussion is a measure of complexity in your mind. Not to mention that your language has been around for nearly a decade while not getting significantly more attention than Jolt. So, not sure that's the comparison you want to be making.
An obvious suggestion is to look at the amount of code and actual functionality implemented. The fiber implementation alone in Jolt is more complex than most of your toy. The fact that you keep doubling down on pretending that these things are in any way comparable is frankly incredible. So, no, there's nothing to retract here. You clearly haven't spent even a few minutes to look at what Jolt actually is and what it does.
bmo | 22 hours ago
If we want to measure "complexity" and especially lines of code, I'm quite sure your agent-produced code is going to win, hands down. Whether that's desirable is a different question.
Yogthos | 22 hours ago
Again, if you think you can write something comparable with less code go ahead and put your money where your mouth is. Also, feel free to look at the amount of code in the official Clojure implementation and the Java standard library. The complexity here is inherent in the problem being solved, and doesn't come from code being agent produced as you're trying to insinuate. Either you genuinely don't understand what you're debating here or you're intentionally trolling. Which is it?
bmo | 22 hours ago
There's a lot of goalpost moving here which doesn't make for a productive discussion, but it's a bit silly to issue a challenge like this - it only takes one counterexample. I think people are capable of a lot more than you imagine they are.
patchunwrap | 19 hours ago
For what it's worth I think your compiler with interop with C is a relatively comparable project. Clojure and Java do certainly have their complexities after 20 years. But so does the C abi.
I don't feel like contributing to what is obviously an unproductive conversation. But I did want to ask, how did you motivate yourself to work on a project like kit? How did you avoid short/long term burnout?
bmo | 19 hours ago
It was a little circuitous. I'm really intrinsically motivated to make games, which is how I got started programming as a kid. Kit was an extreme case of yak shaving where I had been using Haxe which I enjoyed and contributed to, had some ideas on how my workflow could be even better which the Haxe team didn't feel fit into that language, and decided to experiment myself. Eventually I ran into enough pain points that I decided to remove language from my complexity budget and moved to C++ where things are more stable than either Haxe or Kit. Putting the effort into the language and compiler wasn't moving me toward my real goal of making games, but it was a great experience nonetheless, and I'm still using the engine I built during that time, now ported to C++.
Yogthos | 21 hours ago
There aren't any goal posts moving here. What I said originally was that I was able to make complex project in under three months, and I challenged people to try building something on the same scale by hand.
You sealioned into the thread and tried to make an equivalence with your toy compiler. Then when I pointed out that what I built is far more complex, you started saying that I'm moving some unspecified goalposts. It should be obvious to anybody with even a modicum of integrity that you have to compare apples to apples to have a meaningful discussion.
I've been doing software development for over two decades now, and I know there's no way I could write something like this by hand in that amount of time. In fact, there have been plenty of abandoned attempts to port Clojure to run on Scheme over the years, and I'll leave it as an exercise for the reader to figure out why nobody managed to complete this herculean task manually before.
elliotmorris | 20 hours ago
This is the entire problem with making claims like this.
No one can verify if you're correct or not. You are an anonymous poster on the internet making a claim of valuable complexity that indicates a category distinction. If you are correct, and your project is in fact, significantly more complex, then the level of effort to verify that claim is also significantly higher, to the degree that no reasonable person is going to do it.
Therefore, it follows that you can't really claim complexity as any sort of argument when posting anonymously on a public forum without anything else, success, reputation, etc, to back it up. You may know for a fact that you are right, but no one else can.
Yogthos | 19 hours ago
Anybody can download Jolt and run the benchmarks in the repo. Anybody can go to the Jolt page and try running existing Clojure libraries listed there https://jolt-lang.net/docs/libraries.html Anybody can go to the examples repo and run example apps there https://github.com/jolt-lang/examples
Anybody can go to the #jolt channel on Clojurians slack and see what people are currently building with it and what their experience is like. I'm also not posting anonymously. I've been around Clojure community for over a decade now, and I have books published, I maintain a web framework, and have lots of open source libraries with literally millions of users.
So, don't pretend that this is some inscrutable thing that nobody can verify.
elliotmorris | 18 hours ago
I'm not pretending that. I'm just trying to get across, and it's a difficult concept to convey so I will be inaccurate, that the sort of verification you're gesturing at is not necessary for someone who already sees why you're right, and is unlikely to be performed in this context by someone who cannot. Hence, I believe claims presented towards the latter group in this unequivocal way are more or less pointless.
You are right, I was wrong to call you anonymous. The concept I was gesturing at was effectively anonymous for the purpose of how folks are going to consume a claim like this.
Yogthos | 18 hours ago
Again, this isn't some abstract debate. We are talking about a concrete compiler implementation for a well known language. Seems like the problem isn't with demonstrating what I said factually, but that there is a group of people who just want to argue without spending the time to learn the facts. And that's entirely their problem as far as I'm concerned.
I've presented a claim that I can defend on factual basis. If people want to honestly debate this claim they have to spend the time to investigate the facts. As the saying goes, no investigation, no right to speak.
elliotmorris | 18 hours ago
Okay fair enough. I merely worry that this inevitably means that the camp that has the most power to publish in volume will inevitably be the only camp capable of performing speech, regardless of quality distribution, due to the asymmetric effort implied in the verification effort.
Yogthos | 18 hours ago
I don't really follow what you're even trying to argue here. We are discussing two compiler implementations here, and whether they're comparable in terms of the scale of the problem they solve. What does any of this have to do with publishing in volumes?
Seems to me that actually understanding the subject people are attempting to debate is the bare minimum requirement. I'm not sure how you propose having any intelligible discussion where the participants don't even bother having a minimal knowledge of the subject they're discussing.
Student | 17 hours ago
I think where you’re losing your audience is that you’re calling this other person’s language a toy, then failing to substantiate why it’s a toy. No-one is doing Jolt down.
Yogthos | 16 hours ago
If you look at the repo the author linked, it's very clear that it is a small hobby language. It's precisely the kind of thing you could hope to write by hand in 12 weeks or so.
The entire compiler is under 13k loc of Haskell. The type checker looks like the single biggest subsystem in the whole thing at about 3,500 lines. The README is upfront about calling the whole thing pre-alpha with features not fully implemented.
Kit has no persistent data structures, no lazy sequences, no concurrency model, and its metaprogramming is a term rewriting system over the typed AST.
By contrast, Jolt's compiler front end alone in Clojure weighs in at almost exactly the size of Kit's entire compiler. And that's the smallest part of jolt most of which is written in Scheme with over 50k loc. Overall source is literally an order of magnitude that of Kit while Jolt's tests alone are bigger than Kit's whole project.
And it's not just about lines of code, the projects are genuinely in different categories in terms of what each has to implement. Kit is a statically typed systems language that lowers to C through one backend, a unifier, and a small prelude. Jolt is reimplementing the semantics of a mature dynamic language, persistent data structures, dynamic dispatch, a reader, a loader, a namespace system, an nrepl server, FFI, and it has to get all of the semantics right for existing Clojure programs to actually run on it. For perspective, Clojure proper is similarly large with clojure.lang runtime alone being around 42k lines. So, yes, Kit is a toy in comparison.
It's a sophisticated compiler for a hobby project that implements things like type inference and algebraic data types. But it's simply not in the same league as a language like Clojure. And I personally find it insulting that the author tried to make an equivalnce between the two, then continued to double down talking about absurd metrics like github stars as a measure of complexity.
Student | 5 hours ago
I think it would have behoved you to say this all much earlier and to be less thin skinned. Yes Jolt is impressive but I think you might be overly emotionally invested in people perceiving how great it is.
Yogthos | 4 hours ago
I'm not emotionally invested in people perceiving how great it is at all. I'm simply offended that Kit author would make an obvious false equivalence with my project and then refuse to even acknowledge doing that.
What's most disappointing here is how the thread predictably devolved into people just piling on instead of even trying to have an honest discussion on the subject.
The original point I was making is that I was able to make a complex piece of software much faster than I could have done previously. I presented this as something for the skeptics to consider.
And the skeptics predictably started doing mental gymnastics along with making personal attacks and false equivalences. Nobody has so far shown anything even remotely comparable being done by hand on the same time scale.
All this shows to me that people have already made up their minds on the subject, and no amount of evidence will convince them of anything. People made the conclusion that LLM development doesn't work and now it's basically no different from a cult. Rational discussion on the subject is clearly not possible here.
Loup-Vaillant | 25 minutes ago
You can trivially defend achievements and capabilities of your product, you can easily defend its current complexity.
Good luck however defending essential complexity, without putting the burden of proof on the naysayers...
...There you are. But that's not how it works. You can't change people's minds without presenting evidence, and the kind of evidence you have access to with respect to essential complexity is likely very hard to convey.
cpurdy | 4 hours ago
And this is the fundamental problem: You just suggested that someone else invest (possibly waste) their time to validate some giant black box of likely-slop code sprawl that you had a team of GPU-monkeys-with-keyboards spit out.
You're asking for the impossible, like asking someone to review all of the 30,000 minutes of video that is being uploaded to Youtube ... in just the last minute! So if you speed up the video by 3x, and work 24 hours a day with no sleep, you can only fall behind by 43,000,000+ minutes of video every day. This is not a winnable war.
So much of our industry, our relationships, and our society itself is based on trust. The careless use of LLMs has degraded that currency, badly. Perhaps you, for example, carefully instructed the LLM, and personally reviewed, understood, and tested each line of code. Your project isn't slop! It's pristine. It's perfect.
But it's hidden inside 43,000,000 minutes of slop.
Yogthos | 30 minutes ago
No, the fundamental problem is that people want to have opinions on things they haven't spent time investigating. Nobody is asking anyone to write vapid comments on subjects they don't want to spend the time to understand.
Also, we're not talking about some random thing I linked to without any context. We're discussing a project I built. You are talking to the author. There is no reason for me to try to trick you into anything. This project also reflects on my reputation as a developer and I'm putting that on the line by showing it discussing it. The fact that you don't even have the basic decency to acknowledge this says volumes.
My original point was that I made something that solves a non-trivial problem as an example for people who are skeptical about LLMs. Instead of having a rational discussion about the project, this thread turned into personal attacks.
It's pretty clear that there's just an echo chamber of people claiming that LLMs don't work, and when they're presented with counter examples they just glibly dismiss them because apparently it's jut too onerous to actually engage with other developers in good faith for you lot.
alexandria | 12 hours ago
Genuinely, this shows your ignorance of programming as a field. There are many, many examples over the last sixty years of programmers doing the same, or more, in lesser time scales independently. Maybe you should ask your LLM about it.
As someone who has taken the time to learn about the history of programming, your pet project is not particularly impressive, and the fact that you have been very visibly bad-mouthing a programmer who accomplished similar without the use of LLMs reflects very poorly on you, as a programmer, as a professional, and as a person.
Yogthos | 6 hours ago
Genuinely, this shows your ignorance of programming as a field. Feel free to provide examples of people writing a project with the scope of Clojure in a couple of months. As someone who has taken the time to learn about the history of programming, and having done development for decades, I can spot a troll when I see one.
spc476 | 17 hours ago
I tried a challenge the other way and it did not go well at all. The person who tried to LLM the code I failed to write basically didn't understand the problem and didn't even bother to do all that much. Feel free to try either issue I mentioned; it should be even easier a year later with an LLM, right?
Also, it took me two weeks to implement by hand, ANS Forth for the Motorola 6809 [1]. An LLM should be able to chunk that out in a day, right? Feel free to try it. Oh, and my ANS Forth system passes the ANS Forth test suite for the wordsets I did implement. Or is this too trivial a project?
[1] I did not implement BLOCK, FACILITY, FILE, FLOATING, MEMORY or XCHAR (or the extensions) as those were more OS-dependent, took too much memory, or don't make sense on an 8-bit system.
Yogthos | 17 hours ago
I don't really have any context for what the problem being solved there is. I don't see how I'd be able to use LLMs to solve something that's outside my domain of expertise. LLMs aren't magic, they let you work faster, but they don't do thinking for you.
An LLM on its own is not able able to do much of anything, it requires the user to have domain expertise to apply it effectively. The reason I can build a Clojure compiler using these tools is because I have nearly two decades of experience working with the language. I know how it works internally, I know what the end solution needs to look like, and I know pitfalls to avoid.
Trying to solve a problem I have no familiarity with would just be me throwing darts at the board. Maybe the LLM will produce the right solution and maybe it will now. I'm not equipped to evaluate that.
spc476 | 14 hours ago
And your challenge is different how? To take it anyone would need domain knowledge, right? Anyone trying to replicate your results by hand would need it. So is it a valid challenge?
Yogthos | 5 hours ago
My challenge is to build something of comparable scale by hand in the amount of time. It doesn't have to be the same thing I built, but something with a comparable level of complexity in the domain you're comfortable with. Or even better, simply show me a prior example of somebody doing that by hand.
hjvt | 12 hours ago
Imllementing an existing language is obviously the easier task? You have the reference materials, all the PLD decisions have already been made for you.
Yogthos | an hour ago
I encourage you to try actually doing a task on this scale sometime to see whether you find it easy or not. While having lots of reference materials is very helpful, actually doing the work is a very labor intensive process, and putting all these things together in a way that actually works is not easy in any sense. In fact, there have been numerous previous attempts to do this, such as clj-scheme that never got to the point of being generally usable. If this was easy to do as you suggest, we would've seen this done a long time ago.
Making your own compiler with an order of magnitude less code where you can control the scope and make all the decisions is a far easier task.
gonz | 11 hours ago
I have found over an immense amount of cases that people who don't know how to do something tend to overestimate how long they take to make as well as how complex they are. Vibecoders and LLM overusers tend to never actually learn how anything works, so they tend to stay in that zone.
elliotmorris | 7 hours ago
As much as the progenitor of this thread appears to be missing some of the point, I have some doubt that this is the case here, they look to be credentialed in their field, AI usage aside.
We cant be sure of this though, because I at least lack domain knowledge, and the traditional markers of trust have been almost completely eroded by LLMs when it comes to communicating with an audience even slightly outside of any specific domain. This is the point. Claims like this suck now, there's no path in which they accomplish anything.
Must be incredibly frustrating to be on the other side of it as well, assuming good faith. (Which is also much harder to do, given the tooling and the tone.)
Student | 5 hours ago
Leaving aside the “how clever a human you have to be” aspect, there’s a shovelware aspect to Jolt that indubitably would have been very difficult to do in three months along with everything else. A lot of a product being mature is having a bunch of stuff and features. The frontier of what size of standard library a new language or implementation can ship with has definitely shifted.
Yogthos | 8 minutes ago
So, would you refer to Clojure itself as shovelware as well, or are you just admitting that I was able to get Jolt up to parity in a backhanded way here? The fact that LLMs make it possible to get a project to a usable state faster is sort of the whole point here. What I find particularly telling is that there hasn't been a single constructive criticism of Jolt in this whole thread. There are plenty of people slinging mud at me, denigrating me, attacking my character and my work. But not a single comment actually engaging with the quality of the project or the complexity of the problem it solves. That says a lot about the whole skeptic position if you ask me.
cpurdy | 4 hours ago
The number of blog posts celebrating LLMs or conversely explaining away LLMs are legion. The next batch of trained LLMs will now know what they need to know to defend or dismiss LLMs. Mission accomplished.
creesch | 12 hours ago
It is near the bottom but this one is so often overlooked. There are skill plugins like superpowers that attempt to let these models/harnesses make use of good practices like TDD, DRY, etc and they actually do improve the quality output in a remarkable way (there are still plenty of sneaky micro hallucinations, but that is besides the point).
But, none of the frontier labs includes those practices themselves in their harnesses or model training to any significant degree as far as I can tell. They all seem to focus on the sort of quick idea to code type development that works for prototypes but falls apart for larger projects.
Most people I know who use AI successfully in their workflow have had to spend A LOT of time putting up a ton of skill scaffolding and other instructions before they got to that point. Some of that comes down to project specific idiosyncrasies but a majority from what I have seen is the same sort of stuff as the superpowers skill set, trying to get these things to follow basic fundamental software development practices.
Then there is the fact that writing the code is just one aspect of the entire process to begin with. Something I have written about before to the point that the quote I am about to include quotes a previous comment as well, going full matryoshka doll.