I have to say, I'm very interested in the concept of decoherence as a way to explain the reasons why I see AI doesn't really work for creating new code in a way that results in "good enough" code. Please note that I'm putting aside moral considerations to adapt to the style of the discussions present in these forums, but I do have moral concerns with the way that LLMs generate code. Namely that it seems to grab code verbatim from open source code and claim it as its original creation without providing the original author its credit. And other ethical considerations around workers rights, and of course the climate impacts.
But beyond that, LLMs do not produce code that is good enough in cases where collaboration is required. I'm sure at this point we've all seen cases of someone who is a "AI-superstar" excelling at all of the metrics but leading failing projects, because no matter how fast code can be created, the shared understanding we have around the code that we are creating, and why we are writing the code are still important factors. Does it matter if you write code fast, but no-one in your team can understand it and maintain it once you are gone? Does it matter if you write code fast, but the code does not meet the requirements of the user?
In my view, the answer to both of the above is no: having software that aligns to the need of the users is the most important, and software is not useful if it cannot be modified and maintained as needs change. This aligns pretty well with your concept of decoherence. Never mind the impact for organisations larger than a single team.
Namely that it seems to grab code verbatim from open source code and claim it as its original creation without providing the original author its credit.
Have you seen recent examples of this? Early Copilot et al used to do this constantly, I assume because they were small, early models with limited training sets. I haven’t seen a model obviously do something like this in 2026, from any provider I’ve used for work or limited personal use, but if it still happens, I’m interested in seeing what space it happens in.
Either they've gotten better at blocking verbatim code from being output, or the size of the models just make it that instead of stealing a lot from a few projects, they're now stealing a little from a lot of projects.
In my mind that's actually worse than it used to be.
I appreciate you working through this dynamic so gradually, and with so many intuitive hooks. I have been talking about decoherence with colleagues and confidants a lot over the last few years, but it is hard to convey or recognize how dire this problem is, and your sustained exposition helps a lot.
that depends on the nature of the complexity we are facing. Does it:
Grow linearly with the size of the codebase as a whole, OR
Grow exponentially, with each change increasing the likelihood that the next change will add even more complexity.
In my experience, it is the latter.
I think we don't need anecdotes or personal experience to know that in programming (and in most non-trivial systems we deal with) it is the latter, since we know that combinatorial explosion is induced at every step: the number of state in a system produces a state explosion, the number of functions in an API produces an explosion of function compositions, inheritence explosions, explosions between combined state spaces at integration boundaries, of composed sub systems, even just in the naming of things.... like labyrinths where every tile is made up of labyrinths made up of tiles which are all etc.
It is truly a wonder that LLMs have proven such effective tools for getting plausible, and often useful, results out of massive n-dimensional search spaces, but combining their results does indeed produce a decoherence -- we are seeing that nearly everywhere, IMO -- and all the most interesting domains indeed have exponentially growing combinatorial complexity!
I have very much had that same experience so often, and I think it's compounded by the fact that we have so many artificial barriers that force us to reimplement systems in each individual firm, each individual company, etc. Unifying across all those use cases is, of course, a much harder task - but it would be worthwhile compared to the complexity explosion in the medium to long run. I'm going to explore this more in the next post.
It is truly a wonder that LLMs have proven such effective tools for getting plausible, and often useful, results out of massive n-dimensional search spaces
I would actually argue that the reason they are able to is because we have built up so much coherence in these systems! They are super well defined, super well abstracted, and agents combine the power of those existing abstractions really very well.
The issue is, when you try to fill in the space between abstractions, or when you try to push to new layers of abstraction, they can't really handle that complexity, particularly over time.
I would actually argue that the reason they are able to is because we have built up so much coherence in these systems! They are super well defined, super well abstracted, and agents combine the power of those existing abstractions really very well.
That is surely a relevant factor, but people have been exploring automated program synthesis and natural language generation, and image generation, and general problem solving, for a really long time by many different methods, and AFAIK, LLMs have facilitated a notable breakthrough in navigating those search spaces. Arguably having "built up so much coherence" helps reduce the size of available solutions somewhat, but its still the case that quite impressive results are being produced in domains that have massive combinatorial complexity.
Oh yes, I'm not saying that they're not a major breakthrough at all, they definitely are a huge step forward. I'm just saying that so far, they don't seem to be able to build coherence themselves, so they rely on the natural language space that we have built up over the past several millennia. This is why, for instance, you can't train LLMs on their own output.
Very good writing, and I agree with the analysis. I might disagree that this is a bad thing, though. With these tools and the impossibility of having a globally coherent software stack clearly there will be problems, but there are also positive outcomes, meaning more autonomy at the local level because their corner of the maze is inscrutable. For the Engineer's mind this is a net negative, but these tools will enable non-technical people to produce "broken by default" software that serves their purpose without having to cohere with a bigger system as it is now. This means, in many ways, undermining Big Tech and creating more technological autonomy for local communities. Obviously the Engineer is not concerned with this stuff, but it's part of the picture: the monopoly of the Engineer might be over.
Hmm. So, funnily enough, this is close to where I'll end up near the end of my essays, in terms of where I think tech should be going. But the thing is, I think to get there we actually need to push more deeply into better abstractions and a more coherent ecosystem.
I discuss this a bit in my AI usage policy in the section on code. Specifically, the idea behind a universal design system, which would enable much more flexible UIs driven by agentic development, but on much stricter guard rails that would prevent decoherence from seeping into the core layers and leave it just on the surface.
Funnily enough I have a coworker that has been exploring how to keep it at bay, I've been bouncing a lot of ideas with him and exploring the space. He's building something called coherence.
It's mostly explorative and it's absolutely not ready for any type of consumption beyond some internal testing, but so far it works fairly well.
Heya! I used AI for most of them, with a hard constraint on keeping the images simple, informational, and purely SVG based.
I talk about the rules and constraints I put on my usage of AI for visual generation in my AI usage policy. Essentially, only use it for informational purposes, keep it generic, and don't use it for anything I wouldn't have done myself, manually, before. But as I discuss in the guide, this is an area I'm actively evolving my stance on, and I'm generally open to debate around usage for informational purposes.
Whatever system of skills you have developed for this is very high caliber, they feel consistent and are meaningful. Would love a blog post about those if you are taking content requests.
andersmurphy | a day ago
Fun read. It's going to be a real mess, entropy has been massively accelerated.
SamRW | 20 hours ago
I have to say, I'm very interested in the concept of decoherence as a way to explain the reasons why I see AI doesn't really work for creating new code in a way that results in "good enough" code. Please note that I'm putting aside moral considerations to adapt to the style of the discussions present in these forums, but I do have moral concerns with the way that LLMs generate code. Namely that it seems to grab code verbatim from open source code and claim it as its original creation without providing the original author its credit. And other ethical considerations around workers rights, and of course the climate impacts.
But beyond that, LLMs do not produce code that is good enough in cases where collaboration is required. I'm sure at this point we've all seen cases of someone who is a "AI-superstar" excelling at all of the metrics but leading failing projects, because no matter how fast code can be created, the shared understanding we have around the code that we are creating, and why we are writing the code are still important factors. Does it matter if you write code fast, but no-one in your team can understand it and maintain it once you are gone? Does it matter if you write code fast, but the code does not meet the requirements of the user?
In my view, the answer to both of the above is no: having software that aligns to the need of the users is the most important, and software is not useful if it cannot be modified and maintained as needs change. This aligns pretty well with your concept of decoherence. Never mind the impact for organisations larger than a single team.
klardotsh | 6 hours ago
Tangential to your main point, but:
Have you seen recent examples of this? Early Copilot et al used to do this constantly, I assume because they were small, early models with limited training sets. I haven’t seen a model obviously do something like this in 2026, from any provider I’ve used for work or limited personal use, but if it still happens, I’m interested in seeing what space it happens in.
Pandora | 6 hours ago
Either they've gotten better at blocking verbatim code from being output, or the size of the models just make it that instead of stealing a lot from a few projects, they're now stealing a little from a lot of projects.
In my mind that's actually worse than it used to be.
lumi | 5 hours ago
Even if it's not as obvious anymore, it's still a soulless regurgitation machine.
shonfeder | 21 hours ago
I appreciate you working through this dynamic so gradually, and with so many intuitive hooks. I have been talking about decoherence with colleagues and confidants a lot over the last few years, but it is hard to convey or recognize how dire this problem is, and your sustained exposition helps a lot.
I think we don't need anecdotes or personal experience to know that in programming (and in most non-trivial systems we deal with) it is the latter, since we know that combinatorial explosion is induced at every step: the number of state in a system produces a state explosion, the number of functions in an API produces an explosion of function compositions, inheritence explosions, explosions between combined state spaces at integration boundaries, of composed sub systems, even just in the naming of things.... like labyrinths where every tile is made up of labyrinths made up of tiles which are all etc.
It is truly a wonder that LLMs have proven such effective tools for getting plausible, and often useful, results out of massive n-dimensional search spaces, but combining their results does indeed produce a decoherence -- we are seeing that nearly everywhere, IMO -- and all the most interesting domains indeed have exponentially growing combinatorial complexity!
[OP] maevyn | 20 hours ago
Thank you!
I have very much had that same experience so often, and I think it's compounded by the fact that we have so many artificial barriers that force us to reimplement systems in each individual firm, each individual company, etc. Unifying across all those use cases is, of course, a much harder task - but it would be worthwhile compared to the complexity explosion in the medium to long run. I'm going to explore this more in the next post.
I would actually argue that the reason they are able to is because we have built up so much coherence in these systems! They are super well defined, super well abstracted, and agents combine the power of those existing abstractions really very well.
The issue is, when you try to fill in the space between abstractions, or when you try to push to new layers of abstraction, they can't really handle that complexity, particularly over time.
shonfeder | 4 hours ago
That is surely a relevant factor, but people have been exploring automated program synthesis and natural language generation, and image generation, and general problem solving, for a really long time by many different methods, and AFAIK, LLMs have facilitated a notable breakthrough in navigating those search spaces. Arguably having "built up so much coherence" helps reduce the size of available solutions somewhat, but its still the case that quite impressive results are being produced in domains that have massive combinatorial complexity.
[OP] maevyn | 3 hours ago
Oh yes, I'm not saying that they're not a major breakthrough at all, they definitely are a huge step forward. I'm just saying that so far, they don't seem to be able to build coherence themselves, so they rely on the natural language space that we have built up over the past several millennia. This is why, for instance, you can't train LLMs on their own output.
SamRW | 20 hours ago
I never realised complexity in software grew exponentially. I guess it is a really strong argument for keeping things as small as possible!
chobeat | 15 hours ago
Very good writing, and I agree with the analysis. I might disagree that this is a bad thing, though. With these tools and the impossibility of having a globally coherent software stack clearly there will be problems, but there are also positive outcomes, meaning more autonomy at the local level because their corner of the maze is inscrutable. For the Engineer's mind this is a net negative, but these tools will enable non-technical people to produce "broken by default" software that serves their purpose without having to cohere with a bigger system as it is now. This means, in many ways, undermining Big Tech and creating more technological autonomy for local communities. Obviously the Engineer is not concerned with this stuff, but it's part of the picture: the monopoly of the Engineer might be over.
[OP] maevyn | 10 hours ago
Hmm. So, funnily enough, this is close to where I'll end up near the end of my essays, in terms of where I think tech should be going. But the thing is, I think to get there we actually need to push more deeply into better abstractions and a more coherent ecosystem.
I discuss this a bit in my AI usage policy in the section on code. Specifically, the idea behind a universal design system, which would enable much more flexible UIs driven by agentic development, but on much stricter guard rails that would prevent decoherence from seeping into the core layers and leave it just on the surface.
marcecoll | a day ago
Funnily enough I have a coworker that has been exploring how to keep it at bay, I've been bouncing a lot of ideas with him and exploring the space. He's building something called coherence.
It's mostly explorative and it's absolutely not ready for any type of consumption beyond some internal testing, but so far it works fairly well.
ani | 22 hours ago
Apologies that this is off-topic, but I'm fascinated by the illustrations in this essay and I'm wondering how the author prepared them
[OP] maevyn | 22 hours ago
Heya! I used AI for most of them, with a hard constraint on keeping the images simple, informational, and purely SVG based.
I talk about the rules and constraints I put on my usage of AI for visual generation in my AI usage policy. Essentially, only use it for informational purposes, keep it generic, and don't use it for anything I wouldn't have done myself, manually, before. But as I discuss in the guide, this is an area I'm actively evolving my stance on, and I'm generally open to debate around usage for informational purposes.
elliotmorris | 6 hours ago
Really appreciated the thought put into the article. Extra impressive that it's also an example of considered AI usage. Kudos.
ondreian | 22 hours ago
Whatever system of skills you have developed for this is very high caliber, they feel consistent and are meaningful. Would love a blog post about those if you are taking content requests.
wherewhy | 13 hours ago
This was an amazing read – bookmarking this for future reference. The various visualizations are especially highly appreciated!