the process of mastering a difficult field itself is the product, and something that runs is generally a nice-to-have.
Everything about this stance is anathema to the noisy AI boosters:
"gatekeeping" in the form of spending time and effort to learn
focus on journey rather than outcome
usually not directly linked to any profit motive (follows from the runnable code is optional)
Mastery is a deeply meaningful pursuit. I worry we basically ceded the importance of this to those who would rather not put the reps in, and then watch as they try to strong-arm everyone else into pretending that they actually did the reps.
I'll generalise, if I may: creating things is a deeply human pursuit. You don't need to be good at it, and you don't need to be fast at it. Life is about the journey; unfortunately, life under capitalism will have you convinced it's about the end product.
I agree, that's something I've been thinking about. The way AI boosters talk about "being productive" to me seems like deeply internalized trauma from life under capitalism. Protestant work ethic and all that jazz about how work isn't supposed to be enjoyable included.
If it was easy, it wouldn't be interesting; if it wasn't interesting, it wouldn't be worth pursuing as a long-term hobby. The value is inherent in the challenges one overcomes, not in the state of having overcome them. Some people seem to respond to intrinsic motivators, and others extrinsic, and this is IMO the great divide forming in our communities and industries.
Let me explain: Anyone can buy a 3D printer, it's just money. Then they go online and download cool ready made STL models, print a few and go "hmm, these are cool. I'm now the CEO of a 3D printing startup!". Then they order a bunch of fancy filaments and print a massive pile of generic plastic crap and start selling it in every fair and con that they can get a table at.
Meanwhile actual professionals are printing replacement parts for planned obsolesence -inflicted or enshittified equipment or just better versions of existing ones.
The same is happening with LLM dev. The newcomers are SUPER EXCITED that they can HELP by generating a bunch of code (generic plastic dagons) for their favourite projects. Not understanding that it's just shit and nobody really wants it. It's cool if they do it for themselves, but it's not something sustainable.
Meanwhile professionals are using LLMs to enhance their skills, doing the same thing and same quality as before, just a bit faster.
I stopped going to some groups because LLMs converted a bazaar of knowledge gift-giving into a cathedral of "You didn't pray fervently enough or properly to the machine spirit." It converted meetings from a learning community into faith and sales pitches.
I got my career started by going to a Python user group where folks encouraged me, and showed me neat stuff every time. They helped me when I struggled. That's what I don't see anymore at meetups. Instead of it being folks sharing and learning, it's about selling "Look what $AI built for me!"
Since chess engine programming is being mentioned on Lobsters, I think I should comment.
I'm an active chess developer. I have my own engine, have patches in several top engines, including Stockfish, Reckless, and Stormphrax. I have been involved in the development of recent new techniques in the space. (Yes, new techniques to improve chess engines are still actively being discovered after 60+ years!)
1. The (Unintentional?) DoSing of Online Chess Engine Resources
I suspect a lot of the online resources for chess programming are being heavily scraped by bots, because they have gotten significantly less stable and less online over the last couple of years.
The Talk Chess forum which is where the old chess engine guard lives, and while out of date, is also a historical reference.
The new guard and most active chess engine development lives on Discord. While we have felt that this is a negative because it makes modern engine techniques less searchable and we are at the mercy of a corporate entity, the unintentional benefit for us is that this has been a more stable platform for documentation and communication in recent years.
2. The Use of LLMs in Chess Engines
For completeness, I should mention that there have been less controversial experiments of LLMs in engine development. An active and prominent engine developer (tsoj) used LLMs for the automatic generation of new Stockfish search patches. The failure rate of these patches was quite high (90+% failure rate from memory), but this has resulted in a handful of LLM patches in Stockfish.
That said, Stockfish is the strongest engine. I have felt this status of being the strongest has often meant any possible method to improve its Elo has been generally felt to be acceptable; so perhaps this acceptance in SF is not surprising.
3. One Shot LLM Engines
Every now and then someone appears in an engine discord with a engine generated one-shot by an LLM. We do not find this exciting. Personally I feel such engines do not contribute anything new or exciting to the space. Very often these have most of the common search techniques implemented from the "current engine meta" and are very cookie cutter. Often however their performance is significantly subpar compared what you would normally expect for the techniques used, which strongly suggests a lack of proper testing.
I think this is my primary issue with LLMs. Their use by non-experts are often uninteresting and results in subpar work. I personally enjoy living on the frontier. I have inventing new methods or found novel new ways of accelerating things. I feel that my joy of doing so would be sapped by AI, because the joy is in the discovery of the unknown.
4. Coda
Coda was mentioned in the above blogpost. It is an engine that uses current methods for verifying and testing new patches (sequential probability ratio testing) in a framework that is commonly used by engine developers, but new patches are actively scraped from open source engines and fed them into LLMs.
I am just going to copy-and-paste a Discord message I wrote about Coda here:
IANAL but my position is similar to the following:
i think there's both a legal and a moral angle to this:
the legality of strip-mining other engines for ideas and then proceed to copy them, verbatim or otherwise, through a mechanical process
some of the verbatim examples are quite blatant copying without respect to the license
some of them have been LLM-washed in some fashion, however i feel this would would make those patches at minimum derivative works
i feel passing code through an LLM to be rewritten is a mechanical process, and it is up to the courts as to whether that extent of transformation is sufficient for it to be considered derivative
the morality of strip-mining other engines for ideas
i personally don't mind people copying my code as long as a human is learning and carrying forward the knowledge for future developers (I would also like to state however that this is not a legal statement that is not to be taken as the granting of a license), this is part of the reason why i share and document my novelities, because i like sharing and i feel it is the better way for ideas to persist in human minds
the mechanical automation of copying, however, is soul-sapping. i feel less motivated to trial new ideas or come up with things, if they are just going to be put through the copy machine, without at least the bare minimum of another human mind at least going "hah, that's clever!"
i note that coda had copied some of my simd verbatim but reverted it because it couldn't think of how to rewrite it in a fashion that didn't look like blatant copying
i had a few other things i wanted to write but i think those are the main things i can think of at the moment
I don't really have much to add to the above.
Modern engine programming is a generally open community. If you would like to join us, #engine-dev on the Stockfish discord is very active, but there are several other discords where engine developers live, including the Chess Programming Wiki discord and Engine Programming discord.
The summary of the niche communities here feels really foreign to me, given the communities I worked with (os dev, feature phones firmware reverse engineering, esoteric languages). I've never seen gatekeeping - if you found the forum and started doing anything, you were in. Knowledge was as highly regarded as as actually achieving things - some people could reverse engineer and explain lots of things about phones, some just dropped awesome tools for others from time to time. Progress was extremely uneven, mostly driven by when the next really driven person appeared to achieve something weird and exciting.
I wonder if this is a different slice of the communities, or is it just different decades. Honestly, the described hobby communities from the post don't sound fun...
Speaking from the Emu/Homebrew development scene (as a very former homebrew dev, although at least to my knowledge, the emulation scene is better about this), I can recognize a lot of these community traits.
A lot of those communities tend to guard their research obsessively; what's publicly available as documentation is kinda the equivalent of a giant reference dump at the end of the process. Basically you're meant to gawk and be in awe of the hard work the developers are putting in, not actually understand the specifics of how they did that. (Pretty obviously, discord made this issue worse although I'd say the only difference between discord and irc was that the irc channel wasn't archived, so a lot more details could be kept hidden from newcomers.) Trying to learn from public documentation is pretty much impossible; it's pretty much a function dump without any context whatsoever. (Both in RE work and most homebrew libraries.)
The developers of these things tend to deal with so many unskilled users that, rather than pointing people to resources of what they'd need to know, they instead rely on shorthand dismissals meant to dissuade people from working on these projects to begin with. "I want to write a basic program for my 3DS" is met with the "you should learn how to program in C on the desktop first and become experienced in that" response, which is by design very dismissive (because the barrier isn't "learn C", it's "learn C to the point you can use it on a desktop", which is a much higher barrier). Learning how to reverse engineer is given similar barriers. It creates communities that rely on in-groups and access to those in-groups isn't really by technical skills (again, it's a shorthand dismissals, not a genuine assessment of people's skills) but rather by politics/how much you are willing to put up with people being assholes to you in your free time.
Like to give a pretty extreme instance of this; almost none of the c3 talks about the Switch or 3DS led to useful exploits in and of themselves, the ones shown off were just tech demos with the details omitted. The entire reason we had public 3DS homebrew that goes beyond "it can run doom!" was because a different group of people was effectively able to poke through one of the presentations and was able to assemble the incomplete toolchain into a full one with their own research. It took a lot of arguing over the years for transparency to the point where I think the 3DS scene's documentation is now at the point where someone fresh from following a basic C manual can figure out how to write homebrew.
(And that's before we get into things like how a lack of backwards compatibility in a lot of central libraries is used to exclude older projects, which is it's whole other can of worms.)
As for LLMs making those scenes more accessible; I'm ambivalent to it. I obviously don't want to see more AI slop, but I also don't fault people for reaching to the machine that won't permanently judge them as a stupid idiot not worth looking at further, solely because they couldn't figure out the equivalent of learning how the 80 year war worked by only looking at the casualty list.
While I agree with the end of this piece, I feel like (disclaimer: just my personal experience) the gatekeeping and slow pace aren't goals in themselves like this piece indicates, but rather, the result of community design being more of a bottleneck than implementing code. Every design decision you make slightly boxes you in in terms of what you can do in the future without breaking changes, so there's a lot of process around building consensus and making sure there's enough of a shared imagination of where something's going. Just being able to make code faster could possibly just box you in faster without that shared understanding. For technical decisions, that can come from how the code feels when working on it, where LLMs don't give you that same feeling of friction that can help drive discussion. While LLMs can be used by an individual to help understand the code, there's really no substitute for the social exercise of talking to humans in getting consensus on a shared vision.
There's also the bottleneck of code review in open source software projects. LLM usage produces a lot of false positives that are time consuming to debunk. This hits the tightest bottleneck, significantly slowing down the price of progress and grinding down maintainers' will to continue volunteering their time.
Sorry, but no. Hobby open source projects exist largely to scratch some itch. Sure, it can be fun. Sure some started as just a learning experiment or even as a joke. But to jump into the idea that it was mainly about the craft is just a weird and not realistic take.
People put together hobby projects because they could benefit from whatever they have built. Virtually every open source project has some utility and in 99% of the cases, that is the main drive.
Projects reject LLM output not because it robs them of whatever holistic human trait. No one cares about that on the face of an important chunk of progress. They reject it because it's pointless and useless.
Kids go on GitHub, pick an issue, copy paste it in Claude and push the output as a contribution. Why would you think you are adding any value? The owner of the project can do that. Of course the value lies in the control of what gets merged and its quality.
If you think the LLM output is just fine, then why don't you just ask it to implement an alternative to thr project you are contributing to, "but better"? Why would you even contribute to an existing project if the LLM can just create the whole thing for you?
In other words, people created this silly idea that their LLM can just do everything flawlessly no mater the task at hand. Of course experienced engineers wont bother with those believing that.
For hobbyist programming at home, I did explore a bit the options for LLM based programming. The paradox is: LLMs can solve busy and uninteresting work that took me several evenings in minutes. However, it also somewhat defeats the purpose of my hobbyist endeavours.
Like last week I used LLMs to implement a historic programming language, that I didn't find a workable interpreter for. I burnt through like EUR 40 until I had a workable interpreter (WASM-backed). Now I can tinker with the programming language. Sure that is fun, the implementation looks much cleaner than any comparable open source project would look like. but it feels odd to drive a Ferrari to the gym.
mattgreenrocks | 20 hours ago
Everything about this stance is anathema to the noisy AI boosters:
Mastery is a deeply meaningful pursuit. I worry we basically ceded the importance of this to those who would rather not put the reps in, and then watch as they try to strong-arm everyone else into pretending that they actually did the reps.
tentacloids | 12 hours ago
I'll generalise, if I may: creating things is a deeply human pursuit. You don't need to be good at it, and you don't need to be fast at it. Life is about the journey; unfortunately, life under capitalism will have you convinced it's about the end product.
hjvt | 11 hours ago
I agree, that's something I've been thinking about. The way AI boosters talk about "being productive" to me seems like deeply internalized trauma from life under capitalism. Protestant work ethic and all that jazz about how work isn't supposed to be enjoyable included.
sjamaan | 13 hours ago
So basically, AI users are juicers (steroid users)?
tonyg | 7 hours ago
Except that people who use steroids actually get strong!
JulianSildenLanglo | 5 hours ago
Only if you actually exercise!
addison | 4 hours ago
If it was easy, it wouldn't be interesting; if it wasn't interesting, it wouldn't be worth pursuing as a long-term hobby. The value is inherent in the challenges one overcomes, not in the state of having overcome them. Some people seem to respond to intrinsic motivators, and others extrinsic, and this is IMO the great divide forming in our communities and industries.
tmcb | 9 hours ago
shrike | 13 hours ago
LLM dev is in its "3D Printed Dragon" era.
Let me explain: Anyone can buy a 3D printer, it's just money. Then they go online and download cool ready made STL models, print a few and go "hmm, these are cool. I'm now the CEO of a 3D printing startup!". Then they order a bunch of fancy filaments and print a massive pile of generic plastic crap and start selling it in every fair and con that they can get a table at.
Meanwhile actual professionals are printing replacement parts for planned obsolesence -inflicted or enshittified equipment or just better versions of existing ones.
The same is happening with LLM dev. The newcomers are SUPER EXCITED that they can HELP by generating a bunch of code (generic plastic dagons) for their favourite projects. Not understanding that it's just shit and nobody really wants it. It's cool if they do it for themselves, but it's not something sustainable.
Meanwhile professionals are using LLMs to enhance their skills, doing the same thing and same quality as before, just a bit faster.
pyj | 9 hours ago
I stopped going to some groups because LLMs converted a bazaar of knowledge gift-giving into a cathedral of "You didn't pray fervently enough or properly to the machine spirit." It converted meetings from a learning community into faith and sales pitches.
I got my career started by going to a Python user group where folks encouraged me, and showed me neat stuff every time. They helped me when I struggled. That's what I don't see anymore at meetups. Instead of it being folks sharing and learning, it's about selling "Look what $AI built for me!"
87flowers | 8 hours ago
Since chess engine programming is being mentioned on Lobsters, I think I should comment.
I'm an active chess developer. I have my own engine, have patches in several top engines, including Stockfish, Reckless, and Stormphrax. I have been involved in the development of recent new techniques in the space. (Yes, new techniques to improve chess engines are still actively being discovered after 60+ years!)
1. The (Unintentional?) DoSing of Online Chess Engine Resources
I suspect a lot of the online resources for chess programming are being heavily scraped by bots, because they have gotten significantly less stable and less online over the last couple of years.
This includes:
The new guard and most active chess engine development lives on Discord. While we have felt that this is a negative because it makes modern engine techniques less searchable and we are at the mercy of a corporate entity, the unintentional benefit for us is that this has been a more stable platform for documentation and communication in recent years.
2. The Use of LLMs in Chess Engines
For completeness, I should mention that there have been less controversial experiments of LLMs in engine development. An active and prominent engine developer (tsoj) used LLMs for the automatic generation of new Stockfish search patches. The failure rate of these patches was quite high (90+% failure rate from memory), but this has resulted in a handful of LLM patches in Stockfish.
That said, Stockfish is the strongest engine. I have felt this status of being the strongest has often meant any possible method to improve its Elo has been generally felt to be acceptable; so perhaps this acceptance in SF is not surprising.
3. One Shot LLM Engines
Every now and then someone appears in an engine discord with a engine generated one-shot by an LLM. We do not find this exciting. Personally I feel such engines do not contribute anything new or exciting to the space. Very often these have most of the common search techniques implemented from the "current engine meta" and are very cookie cutter. Often however their performance is significantly subpar compared what you would normally expect for the techniques used, which strongly suggests a lack of proper testing.
I think this is my primary issue with LLMs. Their use by non-experts are often uninteresting and results in subpar work. I personally enjoy living on the frontier. I have inventing new methods or found novel new ways of accelerating things. I feel that my joy of doing so would be sapped by AI, because the joy is in the discovery of the unknown.
4. Coda
Coda was mentioned in the above blogpost. It is an engine that uses current methods for verifying and testing new patches (sequential probability ratio testing) in a framework that is commonly used by engine developers, but new patches are actively scraped from open source engines and fed them into LLMs.
I am just going to copy-and-paste a Discord message I wrote about Coda here:
I don't really have much to add to the above.
Modern engine programming is a generally open community. If you would like to join us, #engine-dev on the Stockfish discord is very active, but there are several other discords where engine developers live, including the Chess Programming Wiki discord and Engine Programming discord.
viraptor | 15 hours ago
The summary of the niche communities here feels really foreign to me, given the communities I worked with (os dev, feature phones firmware reverse engineering, esoteric languages). I've never seen gatekeeping - if you found the forum and started doing anything, you were in. Knowledge was as highly regarded as as actually achieving things - some people could reverse engineer and explain lots of things about phones, some just dropped awesome tools for others from time to time. Progress was extremely uneven, mostly driven by when the next really driven person appeared to achieve something weird and exciting.
I wonder if this is a different slice of the communities, or is it just different decades. Honestly, the described hobby communities from the post don't sound fun...
noirscape | 12 hours ago
Speaking from the Emu/Homebrew development scene (as a very former homebrew dev, although at least to my knowledge, the emulation scene is better about this), I can recognize a lot of these community traits.
A lot of those communities tend to guard their research obsessively; what's publicly available as documentation is kinda the equivalent of a giant reference dump at the end of the process. Basically you're meant to gawk and be in awe of the hard work the developers are putting in, not actually understand the specifics of how they did that. (Pretty obviously, discord made this issue worse although I'd say the only difference between discord and irc was that the irc channel wasn't archived, so a lot more details could be kept hidden from newcomers.) Trying to learn from public documentation is pretty much impossible; it's pretty much a function dump without any context whatsoever. (Both in RE work and most homebrew libraries.)
The developers of these things tend to deal with so many unskilled users that, rather than pointing people to resources of what they'd need to know, they instead rely on shorthand dismissals meant to dissuade people from working on these projects to begin with. "I want to write a basic program for my 3DS" is met with the "you should learn how to program in C on the desktop first and become experienced in that" response, which is by design very dismissive (because the barrier isn't "learn C", it's "learn C to the point you can use it on a desktop", which is a much higher barrier). Learning how to reverse engineer is given similar barriers. It creates communities that rely on in-groups and access to those in-groups isn't really by technical skills (again, it's a shorthand dismissals, not a genuine assessment of people's skills) but rather by politics/how much you are willing to put up with people being assholes to you in your free time.
Like to give a pretty extreme instance of this; almost none of the c3 talks about the Switch or 3DS led to useful exploits in and of themselves, the ones shown off were just tech demos with the details omitted. The entire reason we had public 3DS homebrew that goes beyond "it can run doom!" was because a different group of people was effectively able to poke through one of the presentations and was able to assemble the incomplete toolchain into a full one with their own research. It took a lot of arguing over the years for transparency to the point where I think the 3DS scene's documentation is now at the point where someone fresh from following a basic C manual can figure out how to write homebrew.
(And that's before we get into things like how a lack of backwards compatibility in a lot of central libraries is used to exclude older projects, which is it's whole other can of worms.)
As for LLMs making those scenes more accessible; I'm ambivalent to it. I obviously don't want to see more AI slop, but I also don't fault people for reaching to the machine that won't permanently judge them as a stupid idiot not worth looking at further, solely because they couldn't figure out the equivalent of learning how the 80 year war worked by only looking at the casualty list.
Diana | 13 hours ago
It is a different perspective.
From their pov, what you described is gatekeeping, because you need to find the place, and you need to put energy in to be considered in.
I am not saying you are wrong, just... Explaining the pov of the writer.
davepagurek | 20 hours ago
While I agree with the end of this piece, I feel like (disclaimer: just my personal experience) the gatekeeping and slow pace aren't goals in themselves like this piece indicates, but rather, the result of community design being more of a bottleneck than implementing code. Every design decision you make slightly boxes you in in terms of what you can do in the future without breaking changes, so there's a lot of process around building consensus and making sure there's enough of a shared imagination of where something's going. Just being able to make code faster could possibly just box you in faster without that shared understanding. For technical decisions, that can come from how the code feels when working on it, where LLMs don't give you that same feeling of friction that can help drive discussion. While LLMs can be used by an individual to help understand the code, there's really no substitute for the social exercise of talking to humans in getting consensus on a shared vision.
squingo44 | 19 hours ago
There's also the bottleneck of code review in open source software projects. LLM usage produces a lot of false positives that are time consuming to debunk. This hits the tightest bottleneck, significantly slowing down the price of progress and grinding down maintainers' will to continue volunteering their time.
pm | 5 hours ago
Sorry, but no. Hobby open source projects exist largely to scratch some itch. Sure, it can be fun. Sure some started as just a learning experiment or even as a joke. But to jump into the idea that it was mainly about the craft is just a weird and not realistic take.
People put together hobby projects because they could benefit from whatever they have built. Virtually every open source project has some utility and in 99% of the cases, that is the main drive.
Projects reject LLM output not because it robs them of whatever holistic human trait. No one cares about that on the face of an important chunk of progress. They reject it because it's pointless and useless.
Kids go on GitHub, pick an issue, copy paste it in Claude and push the output as a contribution. Why would you think you are adding any value? The owner of the project can do that. Of course the value lies in the control of what gets merged and its quality.
If you think the LLM output is just fine, then why don't you just ask it to implement an alternative to thr project you are contributing to, "but better"? Why would you even contribute to an existing project if the LLM can just create the whole thing for you?
In other words, people created this silly idea that their LLM can just do everything flawlessly no mater the task at hand. Of course experienced engineers wont bother with those believing that.
data_hope | 2 hours ago
At work I code with copilot.
For hobbyist programming at home, I did explore a bit the options for LLM based programming. The paradox is: LLMs can solve busy and uninteresting work that took me several evenings in minutes. However, it also somewhat defeats the purpose of my hobbyist endeavours.
Like last week I used LLMs to implement a historic programming language, that I didn't find a workable interpreter for. I burnt through like EUR 40 until I had a workable interpreter (WASM-backed). Now I can tinker with the programming language. Sure that is fun, the implementation looks much cleaner than any comparable open source project would look like. but it feels odd to drive a Ferrari to the gym.