So I read this title and thought "ah great, yet another article complaining about vibecoded features and their slop". But then I remembered reading some other article about PostgreSQL 19 in the last few weeks and how it claimed that none of its features had been written with the help of LLMs. Which made me curious to read it.
The premise of the article, if I read it right, is that PostgreSQL 19 is late because LLMs found more bugs than would have been found in the past before the release. In my eyes... that's... good? So it's not late because AI is at "fault"; it sounds to me that it's late "thanks" to AI.
Exactly. It means that this iteration of Postgres was forcefully pushed into the “stabilisation” phase, which overall may be a good result for everyone, as a robust base is a better starting point for the new features.
But it obviously frustrates some developers who can not push new stuff as fast as they wanted.
I think in the author's eyes that more bugs were found is also a good thing:
In the end, I don’t think that the delay in shipping PostgreSQL 19 is particularly concerning. It is not a long delay in the grand scheme of things, and it emerges directly from the project’s concern for quality.
Nonetheless, I think he might also be concerned about the consequences of having a potentially huge backlog of new bugfixes that must be checked by a limited amount of human code reviewers.
The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.
As someone who has been loudly in favor of LLMs for quality over velocity, I've really been struggling with a lot of this.
Look, a lot of the systems I help maintain are quite good, right? Not to toot my own horn too much but I've been repeatedly recognized for being able to write solid, high-quality systems. Well, I pointed LLMs at some of those systems and I'm now sitting on several stacks of work, totalling over 100 fixes (before I stopped the LLMs from hunting for even more bugs). I've noticed a few things:
Some of those bugs are quite serious but a lot of them are edge cases that no one is realistically going to hit. The Postgres devs' standards are probably even higher than mine, though.
A lot of the fixes proposed by the LLM are relatively surface-level, and a more systemic understanding would likely result in fewer, better, root-cause fixes. But this kind of analysis is a lot of work that (at least currently) humans have to be in the drivers' seat making judgment calls for.
Every change has a baseline risk -- the systems clearly work well enough today -- and some of the proposed fixes introduce regressions of their own, or at least aren't clearly better than the status quo.
I have so many other things to do!
So I've been very slowly processing the stack of fixes, 3-5 at a time, and not blocking releases on any of it. This is a new kind of judgment, though. In a sense you're effectively DoSing yourself with bug reports and fixes, then figuring out which ones are truly important.
I am wondering:
if we leave security bugs out of the picture, is a bug really a bug if no-one was impacted by it over a long enough time and a wide enough install base?
It is; certainly if it produces observable behaviour that was unintended but people now rely on (Hyrum’s law), and is now blocking features or performance patches, for example.
I know there's a balance to strike, but I think this actually sounds great, especially for something like a database, where reliability is so critical. I wish I could find the blog, but I recall reading a quote recently along the lines of "we're living in a time where we get to choose how many bugs are in our software". While I do think that's slightly hyperbolic, I think the sentiment holds true: we have been given access to some of the best bug finding and fixing tools ever invented. Software engineers have long claimed that we could write bug-free software if given the time -- now is our time!
I know it's hot right now to use LLMs to build every feature imaginable, but I do think that will eventually slow as people get over the hype. In most domains, people will realize that you don't actually want to build every feature. I hope that we can then enter an area of wildly-increased software stability, as well as an era where people start preferring local or E2E-encrypted software, as LLMs find more and more security vulnerabilities in servers.
I wish I could find the blog, but I recall reading a quote recently along the lines of "we're living in a time where we get to choose how many bugs are in our software".
I have mixed feelings about this post and the reactions to it. Over the years I have evolved from "oo, shiney Nosql, that looks new" to "just use Postgres, it's the standard and its solid".
That postgres basically gobbles most new db paradigms, rendering all but the most niche-needy adopters with no choice but to stick with postgres, speaks of a development arc that has constantly added features and complexity. It was inevitable that it was going to reach some sort of "patch criticality" sooner or later.
As a thought experiment, imagine a side universe where LLMs hadn't happened yet. And the postgres guys announced that they'd had it with ever swelling backlogs of bug fixes, and that the next release would be about "stability, refinement, and removing bugs". As software developers, we've cheered this type of move before. Good software engineers have cheered far and wide anytime Apple announces they're just going to do a release that fixes thing.
So this idea that a large project slow the !$%@!$% down and give some attention to stability and quality is not new. It is cool, laudible, that AI "tools" (whatever) are being used to advance this cause for once, but the notion that AI has heralded in something new (an emphasis on quality) is weak, in my opinion. We still (mostly) direct these things. Kudos to the postgres dudes for pointing LLMs at quality, instead of more features (looking at you Particle DB), but the choice to make that the emphasis is about humans. Choices we've had the opportunity to make since long before LLMs arrived on the scene.
One thing that will be interesting to me is the nature of all of these fixes. Like code generation, I find that these tools do certain types will and other parts not as well. They can be OCD about well established anti patterns. But getting them to do "less is more" and innovate so that whole classes of failures go away, that seems to be mostly outside of their ability to introspect and invent still.
jmmv | 15 hours ago
So I read this title and thought "ah great, yet another article complaining about vibecoded features and their slop". But then I remembered reading some other article about PostgreSQL 19 in the last few weeks and how it claimed that none of its features had been written with the help of LLMs. Which made me curious to read it.
The premise of the article, if I read it right, is that PostgreSQL 19 is late because LLMs found more bugs than would have been found in the past before the release. In my eyes... that's... good? So it's not late because AI is at "fault"; it sounds to me that it's late "thanks" to AI.
baggiponte | 12 hours ago
Yeah the title is peak clickbait
thisalex | 12 hours ago
Exactly. It means that this iteration of Postgres was forcefully pushed into the “stabilisation” phase, which overall may be a good result for everyone, as a robust base is a better starting point for the new features.
But it obviously frustrates some developers who can not push new stuff as fast as they wanted.
aggsol | 12 hours ago
Indeed. I would also prefer more bugfixes in pgsql 19 than new features. IMO this is a golden oppurtunity to postpone new features to version 20.
[OP] eduard | 9 hours ago
I think in the author's eyes that more bugs were found is also a good thing:
Nonetheless, I think he might also be concerned about the consequences of having a potentially huge backlog of new bugfixes that must be checked by a limited amount of human code reviewers.
junek | 4 hours ago
radex | 9 hours ago
As much as I dislike LLMs, this seems… fine?
We’re living in the age of slop where velocity is prioritized over quality. Going the other direction is good.
casperin | 5 hours ago
I feel like this is exactly they way it should be done.
sunshowers | 2 hours ago
As someone who has been loudly in favor of LLMs for quality over velocity, I've really been struggling with a lot of this.
Look, a lot of the systems I help maintain are quite good, right? Not to toot my own horn too much but I've been repeatedly recognized for being able to write solid, high-quality systems. Well, I pointed LLMs at some of those systems and I'm now sitting on several stacks of work, totalling over 100 fixes (before I stopped the LLMs from hunting for even more bugs). I've noticed a few things:
So I've been very slowly processing the stack of fixes, 3-5 at a time, and not blocking releases on any of it. This is a new kind of judgment, though. In a sense you're effectively DoSing yourself with bug reports and fixes, then figuring out which ones are truly important.
rameneater | 9 hours ago
I am wondering: if we leave security bugs out of the picture, is a bug really a bug if no-one was impacted by it over a long enough time and a wide enough install base?
tuxes | 8 hours ago
Nobody having reported a bug doesn't mean nobody is impacted.
DB bugs are hard to reproduce because they're often non-deterministic, and only occur in rare conditions.
stig | 8 hours ago
It is; certainly if it produces observable behaviour that was unintended but people now rely on (Hyrum’s law), and is now blocking features or performance patches, for example.
greysonp | 5 hours ago
I know there's a balance to strike, but I think this actually sounds great, especially for something like a database, where reliability is so critical. I wish I could find the blog, but I recall reading a quote recently along the lines of "we're living in a time where we get to choose how many bugs are in our software". While I do think that's slightly hyperbolic, I think the sentiment holds true: we have been given access to some of the best bug finding and fixing tools ever invented. Software engineers have long claimed that we could write bug-free software if given the time -- now is our time!
I know it's hot right now to use LLMs to build every feature imaginable, but I do think that will eventually slow as people get over the hype. In most domains, people will realize that you don't actually want to build every feature. I hope that we can then enter an area of wildly-increased software stability, as well as an era where people start preferring local or E2E-encrypted software, as LLMs find more and more security vulnerabilities in servers.
5d22b | 3 hours ago
https://lobste.rs/s/8izobt/you_can_just_choose_how_many_bugs_you_want
dsschnau | 4 hours ago
It might be helpful to add the 'original' link. - https://neon.com/blog/postgres-19-is-late-and-its-ais-fault
travisgriggs | 2 hours ago
I have mixed feelings about this post and the reactions to it. Over the years I have evolved from "oo, shiney Nosql, that looks new" to "just use Postgres, it's the standard and its solid".
That postgres basically gobbles most new db paradigms, rendering all but the most niche-needy adopters with no choice but to stick with postgres, speaks of a development arc that has constantly added features and complexity. It was inevitable that it was going to reach some sort of "patch criticality" sooner or later.
As a thought experiment, imagine a side universe where LLMs hadn't happened yet. And the postgres guys announced that they'd had it with ever swelling backlogs of bug fixes, and that the next release would be about "stability, refinement, and removing bugs". As software developers, we've cheered this type of move before. Good software engineers have cheered far and wide anytime Apple announces they're just going to do a release that fixes thing.
So this idea that a large project slow the !$%@!$% down and give some attention to stability and quality is not new. It is cool, laudible, that AI "tools" (whatever) are being used to advance this cause for once, but the notion that AI has heralded in something new (an emphasis on quality) is weak, in my opinion. We still (mostly) direct these things. Kudos to the postgres dudes for pointing LLMs at quality, instead of more features (looking at you Particle DB), but the choice to make that the emphasis is about humans. Choices we've had the opportunity to make since long before LLMs arrived on the scene.
One thing that will be interesting to me is the nature of all of these fixes. Like code generation, I find that these tools do certain types will and other parts not as well. They can be OCD about well established anti patterns. But getting them to do "less is more" and innovate so that whole classes of failures go away, that seems to be mostly outside of their ability to introspect and invent still.