Thanks for writing this up. Your background and experience convinced me :-)
There's an obvious incentive for AI companies to credit their own model, over the people whose work the model digested.
If there isn't already, there should be a list of these incidents / marketing programs:
The Navier-Stokes proof and unethical behavior by OpenAI - IIRC they backchanneled an offer to credit one author, as long as they could discredit a rival at Anthropic
(A related issue, which is arguably also bad: "dumping" of hundreds of proofs with questionable citations.)
Anthropic's source control system, as discussed here
I've been using AI to help with a lot of system designs work at $WORK lately (not directly, just "give me ideas for X, pick one, build it out, evaluate whether it works, etc etc etc") and this kind of thing has been on my mind a lot lately: am I ripping someone else off without credit. Like, I think the system I'm building on is different enough from what else is out there that I don't think there is any prior work I could be building on (either in the literature or in trained-on chat messages), but...
I think this is bound to happen, and there are no norms right now. But a clear line is when you start writing a blog post advertising your work, like Anthropic did ...
Oh yeah, if we ever decided to write a blog post or a journal article (this is an area where we could at least plausibly publish a paper) you bet I would do a thorough literature search.
Though IIRC there was some talk I saw implying that this wasn't a case of Anthropic's model reinventing stuff without attribution so much as them deliberately copying existing work without attribution.
I feel like I learned a lot about both Scion and the rest of the source control space from this blog post, especially in comparison to what I learned from the Anthropic one.
Thank you for this write up! I love your writing style and how in depth it is!
I do think the "7 people full time on a major migration (with tokens)" is an interesting strategy. That's probably enough people to really have a good idea of all the parts, that can tackle a lot of different tiny issues, but (if they have the right autonomy) they can get a lot done.
Reading this has lead me to wonder now if we're, just as an industry, trying too hard with monorepos in general though. Like I very much like the consequences of monorepos for working, but it feels like we have so many knock-on effects from trying to work like that.
A part of me wonders if ex-googlers have just kinda poisoned the ranks of staff engineers all over the world and gotten everyone to copy the monorepo + bazel over and over again. I feel like there has to be some more interesting strategies (especially if everyone is doing sparse checkouts anyways!)
But also.... why are there so many merges going on at Anthropic? Are they Meta-sized in terms of amount of disparate projects running with all the random moving parts? Like Meta or Google I feel like there's just so much random stuff going on (the code to the Youtube Music preference pages has to be somewhere....) but I would have imagined that Anthropic it's a bit of a different vibe.
I lack imagination, of course, but it would be funny if all of this is upstream of low merge friction meaning that people have automated jobs just merging in things all the time and creating heavy churn.
Having said all that, the experimentation is neat!
A part of me wonders if ex-googlers have just kinda poisoned the ranks of staff engineers all over the world and gotten everyone to copy the monorepo + bazel over and over again.
I think it's more that if you do multiple repositories you have to figure out policy and tooling for how they're all going to be coordinated. A monorepo means you can just not do that. Having a few folks scale a version control server is a far less terrifying organizational problem then trying to get every team to adopt some particular process and set of tools.
I remember seeing a blogpost that made the claim "if you don't have a monorepo, you don't have 'continuous integration' - you have, at best, 'very frequent integration'"
There are some other negative incentives with a monorepo, which show up whenever Google people touch a project with downstream dependencies. In a monorepo, API providers and API consumers are in the same place. If you want to refactor a component then you can, and then CI will catch any places you failed to fix consumers and you can fix them. Google has some large-scale refactoring tools to help with this. The problem is that making refactoring so easy means that there’s little incentive to think about the impacts of a design: you can always fix problems later. So you end up with more churn. With separate modules that have clearly-defined API contracts, you have to do the design up front because the cost of changing is high. This introduces more friction but also means that you’re in a good position as soon as you need to support an API for external users, because that’s something you’ve been forcing your engineering teams to do all along. To see the impact of this in practice, compare the effort involved in building a ten-year-old Android and iOS app for the latest OS release.
andyc | 16 hours ago
Thanks for writing this up. Your background and experience convinced me :-)
There's an obvious incentive for AI companies to credit their own model, over the people whose work the model digested.
If there isn't already, there should be a list of these incidents / marketing programs:
quasi_qua_quasi | 11 hours ago
I've been using AI to help with a lot of system designs work at $WORK lately (not directly, just "give me ideas for X, pick one, build it out, evaluate whether it works, etc etc etc") and this kind of thing has been on my mind a lot lately: am I ripping someone else off without credit. Like, I think the system I'm building on is different enough from what else is out there that I don't think there is any prior work I could be building on (either in the literature or in trained-on chat messages), but...
andyc | 4 hours ago
I think this is bound to happen, and there are no norms right now. But a clear line is when you start writing a blog post advertising your work, like Anthropic did ...
quasi_qua_quasi | 4 hours ago
Oh yeah, if we ever decided to write a blog post or a journal article (this is an area where we could at least plausibly publish a paper) you bet I would do a thorough literature search.
Though IIRC there was some talk I saw implying that this wasn't a case of Anthropic's model reinventing stuff without attribution so much as them deliberately copying existing work without attribution.
kwas | 13 hours ago
The matter aside, this post is really really well written.
sunshowers | 13 hours ago
Certainly biased here, but seconded. Thanks so much for writing this, ~typesanitizer.
borntyping | 10 hours ago
I feel like I learned a lot about both Scion and the rest of the source control space from this blog post, especially in comparison to what I learned from the Anthropic one.
rtpg | 7 hours ago
Thank you for this write up! I love your writing style and how in depth it is!
I do think the "7 people full time on a major migration (with tokens)" is an interesting strategy. That's probably enough people to really have a good idea of all the parts, that can tackle a lot of different tiny issues, but (if they have the right autonomy) they can get a lot done.
Reading this has lead me to wonder now if we're, just as an industry, trying too hard with monorepos in general though. Like I very much like the consequences of monorepos for working, but it feels like we have so many knock-on effects from trying to work like that.
A part of me wonders if ex-googlers have just kinda poisoned the ranks of staff engineers all over the world and gotten everyone to copy the monorepo + bazel over and over again. I feel like there has to be some more interesting strategies (especially if everyone is doing sparse checkouts anyways!)
But also.... why are there so many merges going on at Anthropic? Are they Meta-sized in terms of amount of disparate projects running with all the random moving parts? Like Meta or Google I feel like there's just so much random stuff going on (the code to the Youtube Music preference pages has to be somewhere....) but I would have imagined that Anthropic it's a bit of a different vibe.
I lack imagination, of course, but it would be funny if all of this is upstream of low merge friction meaning that people have automated jobs just merging in things all the time and creating heavy churn.
Having said all that, the experimentation is neat!
madhadron | 6 hours ago
I think it's more that if you do multiple repositories you have to figure out policy and tooling for how they're all going to be coordinated. A monorepo means you can just not do that. Having a few folks scale a version control server is a far less terrifying organizational problem then trying to get every team to adopt some particular process and set of tools.
jrandomhacker | 4 hours ago
I remember seeing a blogpost that made the claim "if you don't have a monorepo, you don't have 'continuous integration' - you have, at best, 'very frequent integration'"
david_chisnall | an hour ago
There are some other negative incentives with a monorepo, which show up whenever Google people touch a project with downstream dependencies. In a monorepo, API providers and API consumers are in the same place. If you want to refactor a component then you can, and then CI will catch any places you failed to fix consumers and you can fix them. Google has some large-scale refactoring tools to help with this. The problem is that making refactoring so easy means that there’s little incentive to think about the impacts of a design: you can always fix problems later. So you end up with more churn. With separate modules that have clearly-defined API contracts, you have to do the design up front because the cost of changing is high. This introduces more friction but also means that you’re in a good position as soon as you need to support an API for external users, because that’s something you’ve been forcing your engineering teams to do all along. To see the impact of this in practice, compare the effort involved in building a ten-year-old Android and iOS app for the latest OS release.