It's so funny to me how people don't understand that everything connects together. Technology doesn't exist in a vacuum, it's tied to the broader society and economy at large. The reality of the situation is that the statistics show a lagging economy overall in the US, and the money is concentrated in a handful of data centers. So, any disruption to the whole AI build out would almost certainly lead to a major economic crash. And in a situation where you have massive amounts of money being poured into a particular tech that's become central to how the whole economy functions, it should be obvious why a few plucky bloggers aren't going to change the direction of travel.
The author talks about social value, but doesn't seem to realize that the fitness function in a capitalist economy is profit with any social value produced being strictly incidental. There are plenty of enterprises, such as the whole fossil fuel industry, that are actively harmful to humanity. But since they're large and profitable, they continue to flourish in face of that.
The most likely scenario here is that we are in a huge bubble that's driven by capitalist markets, and eventually the bubble will pop leading to another economic crash, which happen roughly every decade under capitalism. At that point the exuberance over AI will fade, but it's highly unlikely the technology itself will simply disappear. I expect that it will be more like the dotcom bubble which created the foundation for the internet today.
I think TFA's argument completely ignores the elephant in the room which is the economic driver behind this technology. And without addressing that, the whole argument becomes divorced from the material reality we inhabit.
The article makes the following statement:
If, however, we can neutralise the LLM threat somehow (perhaps by some combination of regulation, economics and social opprobrium), there is considerable hope.
Yet, here's what empirical research has to say on the subject:
I found this article's list of criteria for "inevitable technology" to make a strong case for coding agents and AI-assisted software development.
The technology can perform a task to a standard equal or not markedly inferior to that of an alternative technology doing the same task. The code I get from coding agents is better than code I can write myself. It's better than most of the code I've seen from collaborators over the course of my career. I think it takes skill to get those results out of a coding agent, but that's why I see coding agents as augmenting, not replacing, skilled programmers.
The task in question is socially valuable: we need a lot of it done, and it's currently done by labour that is, in some sense, scarce. Writing software is very obviously this. Software engineers have always been scarce.
The technology can, in volume of adequate output or in quality of output, overwhelmingly outclass what alternative technologies can do. Absolutely. Armed with a coding agent I can produce thousands of lines of good, tested, understood code in a day (or tens of thousands if I skip the "understood" piece and reduce my criteria for "good"). Without a coding agent I tap out at a few hundred lines per day. A good IDE isn't going to get me the same boost.
There's no way in which the people not using the technology can change their behaviour to neutralise the impact of the technology. This one does have options: regulation. If people not using coding agents can pass or interpret laws such that code by coding agents cannot be sold, can't have copyright applied, or is otherwise illegal, they might be able to stem the tide. At this point that's getting into international treaty territory though, which is still possible - it mostly worked for copyright.
Regulation doesn't need a global treaty, though. A EU-only regulation which makes some kind of liability implicitly dependent on the ML use or lack thereof would already be a strong factor.
The code I get from coding agents is better than code I can write myself. It's better than most of the code I've seen from collaborators over the course of my career. I think it takes skill to get those results out of a coding agent,
So if you are weaker at programming than an LLM, why do you think you are better at prompting LLMs? There are skills required in either case, and from what I've read, it seems it requires just as much attention to detail to keep the LLM from going "off-script" and time that used to be spent getting code past the compiler (or unit tests) is now spent getting code past the LLM.
I can produce thousands of lines of good, tested, understood code in a day
Well, "good" and "understood" are doing some heavy lifting there. From what I've read, the moves made by AlphaZero (Google's AI Go playing engine) are upsetting to Go masters because the moves are rarely understood by them. While AlphaZero plays by the rules, the "output" (so to speak) is so alien that it might as well be random moves that just happen to win. It's hard to determine if the moves are good if the reasoning behind them are not understood.
On your coding thousands of lines of code, over the past forty years we've slowly hammered into upper management that "lines of code" is not a good metric, and now that's suddenly tossed out the window and "more more more" is the mantra of the day? I thought "no code" was the best code.
Without a coding agent I tap out at a few hundred lines per day.
The last year at my previous job, I wrote less than 100 lines of code, and it wasn't because I'm a slow typist. It wasn't even because I wrote 10,000 lines then deleted 9,900 of them. No, it was the constant meetings that slowed the pace down for everybody!
Without a coding agent I tap out at a few hundred lines per day.
The last year at my previous job, I wrote less than 100 lines of code, and it wasn't because I'm a slow typist. It wasn't even because I wrote 10,000 lines then deleted 9,900 of them. No, it was the constant meetings that slowed the pace down for everybody!
And given that many people read emails during those meetings but try not to type full-speed all the time, automatic code generation produces something to review during a pointless meeting. What company does is dictated by how internal communications are structured, a version of Conway's Law…
we've slowly hammered into upper management that "lines of code" is not a good metric, and now that's suddenly tossed out the window and "more more more" is the mantra of the day
I understand the objection but between developers we can share a more nuanced view. If you’re not putting out fires, having meetings, or helping teammates - actively implementing something - and past the phase where you’re figuring out what you’re going to do in the first place, then over time we do learn the baseline speed at which we emit decent production code.
My numbers are similar, 100-200 lines per half-day session. If in that same session under similar circumstances you get 5x as much quantity with similar quality, then that is remarkable. That’s why it gets brought up, for lack of a better objective measure.
such that code by coding agents cannot be sold, can't have copyright applied, or is otherwise illegal
I think in practice only the third one would really work. I know there's a whole section of the market working on software products sold to others. But there are many like me who either work on open source or get paid to create specific software for a company. Practically nothing would change for me if I was forced to say "this is public domain" instead of "this is MIT / proprietary-by-default". The only reason I'm creating proprietary code is the default copyright - there's no reason anyone would want a copy of this hyper-specific implementation.
Somewhat weird verbiage. AI is very much already here. If the author thinks it will not persist, it would be more accurate to say "AI is not here to stay", no?
Most human work, however, is not done at a good standard. At best it's mediocre, and at worst it's downright horrible.
It's annoying when people consciously decide to put their line of "good" somewhere in the clouds where according to their line ~everything sucks and ~nothing is good. I think most of the work I see done is done to a good standard and insisting otherwise is an insult to the people doing it and not a good way to define good and bad.
As to whether modern AI is performing to the "highest of human standards" and makes an "overwhelming" difference or is merely "superficial", can we look, like, two to five inches in front of our nose? The level of performance is a line that has been accelerating up and up for the past 5 years, from the very bottom of human level to, at least, near the top. Nobody in the field sees any reason for it to stop going up. So why write an extended argument with the implicit and undefended premise of "if it stays basically the way it is exactly this week"? Presumably that's not what people are envisioning when they say it's "inevitable", right?
Yogthos | 9 hours ago
It's so funny to me how people don't understand that everything connects together. Technology doesn't exist in a vacuum, it's tied to the broader society and economy at large. The reality of the situation is that the statistics show a lagging economy overall in the US, and the money is concentrated in a handful of data centers. So, any disruption to the whole AI build out would almost certainly lead to a major economic crash. And in a situation where you have massive amounts of money being poured into a particular tech that's become central to how the whole economy functions, it should be obvious why a few plucky bloggers aren't going to change the direction of travel.
The author talks about social value, but doesn't seem to realize that the fitness function in a capitalist economy is profit with any social value produced being strictly incidental. There are plenty of enterprises, such as the whole fossil fuel industry, that are actively harmful to humanity. But since they're large and profitable, they continue to flourish in face of that.
The most likely scenario here is that we are in a huge bubble that's driven by capitalist markets, and eventually the bubble will pop leading to another economic crash, which happen roughly every decade under capitalism. At that point the exuberance over AI will fade, but it's highly unlikely the technology itself will simply disappear. I expect that it will be more like the dotcom bubble which created the foundation for the internet today.
mtset | 8 hours ago
Do you think TFA's argument is that "the technology itself will simply disappear"? Why?
Yogthos | 3 hours ago
I think TFA's argument completely ignores the elephant in the room which is the economic driver behind this technology. And without addressing that, the whole argument becomes divorced from the material reality we inhabit.
The article makes the following statement:
Yet, here's what empirical research has to say on the subject:
simonw | 7 hours ago
I found this article's list of criteria for "inevitable technology" to make a strong case for coding agents and AI-assisted software development.
k749gtnc9l3w | 6 hours ago
Regulation doesn't need a global treaty, though. A EU-only regulation which makes some kind of liability implicitly dependent on the ML use or lack thereof would already be a strong factor.
simonw | 4 hours ago
Yeah, that's a good point. The EU have enough collective weight that they could have a material impact here.
spc476 | an hour ago
So if you are weaker at programming than an LLM, why do you think you are better at prompting LLMs? There are skills required in either case, and from what I've read, it seems it requires just as much attention to detail to keep the LLM from going "off-script" and time that used to be spent getting code past the compiler (or unit tests) is now spent getting code past the LLM.
Well, "good" and "understood" are doing some heavy lifting there. From what I've read, the moves made by AlphaZero (Google's AI Go playing engine) are upsetting to Go masters because the moves are rarely understood by them. While AlphaZero plays by the rules, the "output" (so to speak) is so alien that it might as well be random moves that just happen to win. It's hard to determine if the moves are good if the reasoning behind them are not understood.
On your coding thousands of lines of code, over the past forty years we've slowly hammered into upper management that "lines of code" is not a good metric, and now that's suddenly tossed out the window and "more more more" is the mantra of the day? I thought "no code" was the best code.
The last year at my previous job, I wrote less than 100 lines of code, and it wasn't because I'm a slow typist. It wasn't even because I wrote 10,000 lines then deleted 9,900 of them. No, it was the constant meetings that slowed the pace down for everybody!
k749gtnc9l3w | an hour ago
And given that many people read emails during those meetings but try not to type full-speed all the time, automatic code generation produces something to review during a pointless meeting. What company does is dictated by how internal communications are structured, a version of Conway's Law…
thombles | an hour ago
I understand the objection but between developers we can share a more nuanced view. If you’re not putting out fires, having meetings, or helping teammates - actively implementing something - and past the phase where you’re figuring out what you’re going to do in the first place, then over time we do learn the baseline speed at which we emit decent production code.
My numbers are similar, 100-200 lines per half-day session. If in that same session under similar circumstances you get 5x as much quantity with similar quality, then that is remarkable. That’s why it gets brought up, for lack of a better objective measure.
viraptor | 3 hours ago
I think in practice only the third one would really work. I know there's a whole section of the market working on software products sold to others. But there are many like me who either work on open source or get paid to create specific software for a company. Practically nothing would change for me if I was forced to say "this is public domain" instead of "this is MIT / proprietary-by-default". The only reason I'm creating proprietary code is the default copyright - there's no reason anyone would want a copy of this hyper-specific implementation.
yshui | 8 hours ago
Somewhat weird verbiage. AI is very much already here. If the author thinks it will not persist, it would be more accurate to say "AI is not here to stay", no?
mqp | an hour ago
It's annoying when people consciously decide to put their line of "good" somewhere in the clouds where according to their line ~everything sucks and ~nothing is good. I think most of the work I see done is done to a good standard and insisting otherwise is an insult to the people doing it and not a good way to define good and bad.
As to whether modern AI is performing to the "highest of human standards" and makes an "overwhelming" difference or is merely "superficial", can we look, like, two to five inches in front of our nose? The level of performance is a line that has been accelerating up and up for the past 5 years, from the very bottom of human level to, at least, near the top. Nobody in the field sees any reason for it to stop going up. So why write an extended argument with the implicit and undefended premise of "if it stays basically the way it is exactly this week"? Presumably that's not what people are envisioning when they say it's "inevitable", right?