Version control software is something that you really want to be reliable.
Anthropic has a released a lot of vibeslop, and a lot of it is pretty bad. Claude Code has an endless series of terminal-drawing bugs. And even when they're fixed, they come back a week later. This is not the sort of thing that I want in my version control system.
Git, meanwhile, was designed to handle Linux. Linux is a huge project with thousands of contributors and a high rate of merges. Granted, Linux is text, and Git is much weaker for projects with significant non-text data.
But even so, if your problem is that you are generating PRs so fast that you can't keep up with raw git merge operations, then you're either committing a lot of binary data, or you're doing something really weird. (EDIT: Apparently, once Google got up to around 80,000 human engineers commiting to one monorepo, they did need to merge several times per second.)
Then again, one of the most annoying habits of unsupervised LLMs is their tendency to turn what would be a lovely 20,000 line Rust project into 350,000 lines of buggy Go full of 3 subtly different versions of common functions.
one of the most annoying habits of unsupervised LLMs
The key word is unsupervised. If a person writes down the detailed requirements, and ensures that the LLMs follow a good design, it usually ends up with only 60,000 lines of buggy Go full of 3 subtly different versions of common functions. :(
I would’ve loved to see the landscape review they did of other VCSes before building their own. Cynically, yes of course this is content marketing to show off for investors and customers. But even taking it as content marketing I’d expect to see a comparison with other solutions. Let alone taking it at face value as a technical blog post, which should definitely include a discussion of tradeoffs and build vs. buy.
I don't know of any open source VCS systems designed to handle that scale. Commercially there's Perforce, but how does pricing work for that? If it's licensed per install then a million agents is going to be too expensive even for a company with billions of dollars.
(I just had a dig around, Meta's Sapling and Microsoft's Scalar look relevant here. Also I think Perforce is still per-employee and not per-agent, so probably a credible option here.)
Source control for huge monorepos has prior art, notably Google’s Piper and Meta’s Sapling, so we were able to build Scion with their hard-won lessons in mind.
Modern AI-assisted/generated software is bespoke, it makes no sense for them to buy any solution or even compare to one when they can simply build whatever fits their needs precisely.
The average SME likely doesn't have enough people to "AI prompt" all of the software that they need. The ongoing maintenance has a non-zero personal cost, which is likely larger than just buying the software.
I'm not so sure it works even if you have a billion dollars. Claude Code is a bug riddled mess - e.g. on my Mac, I've never been able to copy more than one line of output. This is arguably fine for Claude Code since clearly people are willing to tolerate it (although I think they would improve the company value quite a bit by focusing on fixing basic functionality like copy/paste for a product that's used by millions of users, but whatever.) But applying this same laissez faire approach to a VCS seems like a disaster waiting to happen.
Moreover, a VCS is so core to how an engineering firm works, that any technical debt accrued there has a multiplicative effect due to other infrastructure's reliance on it. I know this because I work for Google, which has its own VCS and therefore necessarily has its own code review system, CI, deployment mechanisms, etc.
You are of course right there’s a cost, but it sounds like alternative cost you have in mind is just the fees for SaaS. To me that’s not the cost to avoid.
I’m setting up a new business, and I am making a lot of choices on build v buy. The choice on cost is simple - the monthly fee for something like Linear is much less than the hourly cost of me working through a solution; linear is much cheaper.
But: Linear means I’m stuck with a generic solution meant for generic tech companies, with lots of baked-in choices I need to accept, tons of missing features I want and thousands of features I don’t want.
Instead, I started with a super super minimal board; a GitHub repo with a template for markdown files, each a ticket, in one of three directories. As I expanded and found new needs I incrementally expanded functionality.
Now each ticket contains active time, lead time and cost estimates. A web interface using the git repo as a database runs Monte Carlo simulations over it to get ranges of possible financial and time scenarios, and can model things like “how would the outcome move of I hired another engineer” or “what if this supplier slips”.
It then integrates - read only - with my accounting system, so the forecasts can be compared to actual costs and actual delivery times.
This is hyper-specialised for my need as a hardware company where much or the work is in the intersection of software planning, certifications and coordinating supply chains.
This kind of hyper-specialised tooling is the value to me, why would I pay for a generic SaaS solution with a bunch of features I don’t need when I can get a solution bespoke for my companies needs?
This kind of hyper-specialised tooling is the value to me, why would I pay for a generic SaaS solution with a bunch of features I don’t need when I can get a solution bespoke for my companies needs?
Very true.
If the AI tools take you $X to build and maintain, you're not going to pay >$X for a commercial product to do the same thing.
My concern is that in many cases, the cost of the AI tools ends up growing over time. This is the usual outcome of "I'll build it myself versus buy it" process. The AI tools drastically lower the cost of "I'll built it myself", but they don't take it to zero. And the maintenance cost of a locally built solution adds up over time.
Sounds like we're on the same page. There's a kind of "shape of thing" emerging in my mind here where some solutions are relatively easy to see that they work right from the output, the model I want is clear to me, and my companies needs have large upside to bespoke solutions (eg. integrates with our specific accounting system, my preference for kanban-style boards, our business need to combine software planning with external delays and deadlines for hardware production)
But then there are other cases where the cost of rebuilding is huge - the accounting system is a good example. Don't get me wrong, out accounting system is a disaster and I want someone to build something better, but it's really, really important to me that the software here is certified, carefully developed, has lots of safeguards, uses standard modelling and so on.
In other words: Replacing the feature set I use in Linear took a half a day of work, with another 3-4 days of work to expand it to something far beyond what Linear gives me. But I would never replace the certified accounting system with something vibe-coded in house, or the version control system, email etc etc.
But there's a whole category of B2B SaaS I think I'll never end up buying again, for better or worse.
And still you need to know what to ask for, and that's what a landscape review is for: Figure out all the dead ends other's have already pursued, lest you think they're actually good ideas and ask your code slave to implement them for you, bespoke and perfectly suited to your whims (not needs).
Which it'll do, sure, but it will still waste your time when you need to dig yourself out of that hole that you asked for.
wait what? Are you saying that we shouldn't use existing tools / libraries / frameworks / operating systems anymore, because we can just build it all from scratch?
I read it; i thought it was pretty neat. I appreciated them sharing the data.
A little voice in my head was saying: "I guess we are back at Perforce" but that was me just being facetious
To my mind, this is speed overriding architecture.
They have a CI/CD problem. CI/CD is a process that almost always has a large number of inefficiencies. But instead of reviewing and fixing the inefficiencies, they seem to have taken the largest and most carefully thought-through part of the process, version control, and rewritten that. Apparently leaving the rest of the teetering pile of CI/CD alone.
How do you end up with a backlog of 30,000 code changes per day though?
It must be unsupervised agents looping through removing and adding the same lines over and over lol
The diagrams in Anthropic blog posts always look nice, notably compared to many vibe-generated diagrams generated by Claude we see everywhere. They don't stuff a lot of text, the arrows are nicely drawn, the text does not overflow, the layout is balanced. I suspect this is something like Mermaid behind the scene, and not a direct generation to SVG, but I would be interested to know more about this.
About the post itself, for me, it's just marketing. If they were really standing behind their solution, they would have published it.
Forget about the source control. What do these companies use as their build system? Surely none of them is on the dumpster fire that is Github Actions?
Another thought is that maybe being able to build and ship systems at this level of scale/novelty is now becoming part of the expectations for a senior+ engineer.
Ok now do one for the code review tool, antique! Bottlenecking on code reviews is much more common at every company scale than bottlenecking on monorepo checkouts :)
Interestingly, it sounds like their model allows for conflicts and doesn't gate the changes. It sounds like a nightmare for reliability, but I guess they are okay with it?
lets pretend that this is actually how they work. if a time-based bug rears its head (say it takes 7 days after deploy to hit the bug), their codebase has now advanced around 200k commits from when the bug was introduced. do we believe that trawling through the old versions to identify the bug is possible here? and at this point, does having version control even matter?
You don't need a codebase of this size to make version history a theoretical afterthought. It's wild how infrequently this "feature" is used at any company.
I mean, if you look at the fact that Git has not turn-key option to sync reflog, you should already make the conclusion that reading history is not a thing.
Blindly bisecting — well, it's logarithmic, so neither size nor quality of the history matters as much, as long as commits are small enough…
But yeah, when we migrated Mercurial-to-Monotone we looked at migration then said «nah we don't care just copy the last snapshot and backup the old repo, we'll dig it up if we ever need it». We did not in fact ever need it.
Pretty much. At my last job, in 12 years, I think I only looked into the history twice while tracking down a bug to see how it got introduced. Weird, [lead developer] wouldn't have made this stupid of a mistake ... oooh! It was the manager! (both times)
I don't think they give enough details to know if it is actually a different approach between picking and combining a few pretty well-known and tried things.
almost 90% of git users doesn't really do a p2p dvcs push/pull, it's all connected to a central github. It's aways easier to optimized a centralized server than full dvcs.
emk | 18 hours ago
Version control software is something that you really want to be reliable.
Anthropic has a released a lot of vibeslop, and a lot of it is pretty bad. Claude Code has an endless series of terminal-drawing bugs. And even when they're fixed, they come back a week later. This is not the sort of thing that I want in my version control system.
Git, meanwhile, was designed to handle Linux. Linux is a huge project with thousands of contributors and a high rate of merges. Granted, Linux is text, and Git is much weaker for projects with significant non-text data.
But even so, if your problem is that you are generating PRs so fast that you can't keep up with raw
git mergeoperations, then you're either committing a lot of binary data, or you're doing something really weird. (EDIT: Apparently, once Google got up to around 80,000 human engineers commiting to one monorepo, they did need to merge several times per second.)Then again, one of the most annoying habits of unsupervised LLMs is their tendency to turn what would be a lovely 20,000 line Rust project into 350,000 lines of buggy Go full of 3 subtly different versions of common functions.
alandekok | 15 hours ago
The key word is unsupervised. If a person writes down the detailed requirements, and ensures that the LLMs follow a good design, it usually ends up with only 60,000 lines of buggy Go full of 3 subtly different versions of common functions. :(
prime | 19 hours ago
I would’ve loved to see the landscape review they did of other VCSes before building their own. Cynically, yes of course this is content marketing to show off for investors and customers. But even taking it as content marketing I’d expect to see a comparison with other solutions. Let alone taking it at face value as a technical blog post, which should definitely include a discussion of tradeoffs and build vs. buy.
simonw | 8 hours ago
Presumably that was a pretty small pool?
I don't know of any open source VCS systems designed to handle that scale. Commercially there's Perforce, but how does pricing work for that? If it's licensed per install then a million agents is going to be too expensive even for a company with billions of dollars.
(I just had a dig around, Meta's Sapling and Microsoft's Scalar look relevant here. Also I think Perforce is still per-employee and not per-agent, so probably a credible option here.)
quad | 6 hours ago
anex9d | 15 hours ago
Modern AI-assisted/generated software is bespoke, it makes no sense for them to buy any solution or even compare to one when they can simply build whatever fits their needs precisely.
alandekok | 15 hours ago
That works if you have billions of dollars.
The average SME likely doesn't have enough people to "AI prompt" all of the software that they need. The ongoing maintenance has a non-zero personal cost, which is likely larger than just buying the software.
m0th | 8 hours ago
I'm not so sure it works even if you have a billion dollars. Claude Code is a bug riddled mess - e.g. on my Mac, I've never been able to copy more than one line of output. This is arguably fine for Claude Code since clearly people are willing to tolerate it (although I think they would improve the company value quite a bit by focusing on fixing basic functionality like copy/paste for a product that's used by millions of users, but whatever.) But applying this same laissez faire approach to a VCS seems like a disaster waiting to happen.
Moreover, a VCS is so core to how an engineering firm works, that any technical debt accrued there has a multiplicative effect due to other infrastructure's reliance on it. I know this because I work for Google, which has its own VCS and therefore necessarily has its own code review system, CI, deployment mechanisms, etc.
tobin_baker | 5 hours ago
Both Claude Code and Codex are so buggy that they seem to serve as cautionary tales for vibecoding rather than demonstrations of its potential.
jjdh | 13 hours ago
You are of course right there’s a cost, but it sounds like alternative cost you have in mind is just the fees for SaaS. To me that’s not the cost to avoid.
I’m setting up a new business, and I am making a lot of choices on build v buy. The choice on cost is simple - the monthly fee for something like Linear is much less than the hourly cost of me working through a solution; linear is much cheaper.
But: Linear means I’m stuck with a generic solution meant for generic tech companies, with lots of baked-in choices I need to accept, tons of missing features I want and thousands of features I don’t want.
Instead, I started with a super super minimal board; a GitHub repo with a template for markdown files, each a ticket, in one of three directories. As I expanded and found new needs I incrementally expanded functionality.
Now each ticket contains active time, lead time and cost estimates. A web interface using the git repo as a database runs Monte Carlo simulations over it to get ranges of possible financial and time scenarios, and can model things like “how would the outcome move of I hired another engineer” or “what if this supplier slips”.
It then integrates - read only - with my accounting system, so the forecasts can be compared to actual costs and actual delivery times.
This is hyper-specialised for my need as a hardware company where much or the work is in the intersection of software planning, certifications and coordinating supply chains.
This kind of hyper-specialised tooling is the value to me, why would I pay for a generic SaaS solution with a bunch of features I don’t need when I can get a solution bespoke for my companies needs?
alandekok | 10 hours ago
Very true.
If the AI tools take you $X to build and maintain, you're not going to pay >$X for a commercial product to do the same thing.
My concern is that in many cases, the cost of the AI tools ends up growing over time. This is the usual outcome of "I'll build it myself versus buy it" process. The AI tools drastically lower the cost of "I'll built it myself", but they don't take it to zero. And the maintenance cost of a locally built solution adds up over time.
jjdh | 10 hours ago
Sounds like we're on the same page. There's a kind of "shape of thing" emerging in my mind here where some solutions are relatively easy to see that they work right from the output, the model I want is clear to me, and my companies needs have large upside to bespoke solutions (eg. integrates with our specific accounting system, my preference for kanban-style boards, our business need to combine software planning with external delays and deadlines for hardware production)
But then there are other cases where the cost of rebuilding is huge - the accounting system is a good example. Don't get me wrong, out accounting system is a disaster and I want someone to build something better, but it's really, really important to me that the software here is certified, carefully developed, has lots of safeguards, uses standard modelling and so on.
In other words: Replacing the feature set I use in Linear took a half a day of work, with another 3-4 days of work to expand it to something far beyond what Linear gives me. But I would never replace the certified accounting system with something vibe-coded in house, or the version control system, email etc etc.
But there's a whole category of B2B SaaS I think I'll never end up buying again, for better or worse.
quad | 6 hours ago
Very cool! Would you be willing to share any of this publicly?
pgeorgi | 14 hours ago
And still you need to know what to ask for, and that's what a landscape review is for: Figure out all the dead ends other's have already pursued, lest you think they're actually good ideas and ask your code slave to implement them for you, bespoke and perfectly suited to your whims (not needs).
Which it'll do, sure, but it will still waste your time when you need to dig yourself out of that hole that you asked for.
skade | 15 hours ago
This is a great at organisations that can state their needs precisely and track the execution and the meeting of those goals accurately.
Problem is that precisely the above is one of the hardest problems we have in software or even in general and has tons of research behind it.
amw-zero | 3 hours ago
wait what? Are you saying that we shouldn't use existing tools / libraries / frameworks / operating systems anymore, because we can just build it all from scratch?
zg | 17 hours ago
I can't rationalize why they write these long, technical blog posts without sharing anything of what they've written. What are they afraid of?
Meanwhile, I can't believe I'm saying this, but I have to give Meta credit for making a similar post, but not being afraid to share it openly: https://engineering.fb.com/2022/11/15/open-source/sapling-source-control-scalable/, https://github.com/facebook/sapling
fzakaria | 18 hours ago
I read it; i thought it was pretty neat. I appreciated them sharing the data. A little voice in my head was saying: "I guess we are back at Perforce" but that was me just being facetious
tallen | 15 hours ago
To my mind, this is speed overriding architecture.
They have a CI/CD problem. CI/CD is a process that almost always has a large number of inefficiencies. But instead of reviewing and fixing the inefficiencies, they seem to have taken the largest and most carefully thought-through part of the process, version control, and rewritten that. Apparently leaving the rest of the teetering pile of CI/CD alone.
sfkvso | 14 hours ago
How do you end up with a backlog of 30,000 code changes per day though? It must be unsupervised agents looping through removing and adding the same lines over and over lol
vbernat | 13 hours ago
The diagrams in Anthropic blog posts always look nice, notably compared to many vibe-generated diagrams generated by Claude we see everywhere. They don't stuff a lot of text, the arrows are nicely drawn, the text does not overflow, the layout is balanced. I suspect this is something like Mermaid behind the scene, and not a direct generation to SVG, but I would be interested to know more about this.
About the post itself, for me, it's just marketing. If they were really standing behind their solution, they would have published it.
alper | 12 hours ago
Forget about the source control. What do these companies use as their build system? Surely none of them is on the dumpster fire that is Github Actions?
Another thought is that maybe being able to build and ship systems at this level of scale/novelty is now becoming part of the expectations for a senior+ engineer.
codekobold | 16 hours ago
Ok now do one for the code review tool, antique! Bottlenecking on code reviews is much more common at every company scale than bottlenecking on monorepo checkouts :)
agent281 | 15 hours ago
Interestingly, it sounds like their model allows for conflicts and doesn't gate the changes. It sounds like a nightmare for reliability, but I guess they are okay with it?
op | 13 hours ago
lets pretend that this is actually how they work. if a time-based bug rears its head (say it takes 7 days after deploy to hit the bug), their codebase has now advanced around 200k commits from when the bug was introduced. do we believe that trawling through the old versions to identify the bug is possible here? and at this point, does having version control even matter?
alper | 12 hours ago
You don't need a codebase of this size to make version history a theoretical afterthought. It's wild how infrequently this "feature" is used at any company.
k749gtnc9l3w | 12 hours ago
I mean, if you look at the fact that Git has not turn-key option to sync reflog, you should already make the conclusion that reading history is not a thing.
Blindly bisecting — well, it's logarithmic, so neither size nor quality of the history matters as much, as long as commits are small enough…
But yeah, when we migrated Mercurial-to-Monotone we looked at migration then said «nah we don't care just copy the last snapshot and backup the old repo, we'll dig it up if we ever need it». We did not in fact ever need it.
spc476 | 5 hours ago
Pretty much. At my last job, in 12 years, I think I only looked into the history twice while tracking down a bug to see how it got introduced. Weird, [lead developer] wouldn't have made this stupid of a mistake ... oooh! It was the manager! (both times)
pronoiac | 6 hours ago
30,000 merges per day is roughly a merge every 3 seconds.
prime | 19 hours ago
iikku | 16 hours ago
It's a truly different approach to version control compared to git or other dvcs's, which I respect
k749gtnc9l3w | 11 hours ago
I don't think they give enough details to know if it is actually a different approach between picking and combining a few pretty well-known and tried things.
pepegar | 16 hours ago
From what I've read online, similar to what they do at Google with Piper, right?
mayli | 4 hours ago
almost 90% of git users doesn't really do a p2p dvcs push/pull, it's all connected to a central github. It's aways easier to optimized a centralized server than full dvcs.