Isn't this going to introduce the dependency of domain management? I understand the positive side of it, but put some infra level manamgment layer to individual Open Source devs, just a thought. But yeah, positives vs negatives weigh and pick.
GitHub is almost forever. Your custom domain disappears when you stop paying the bills, which if you’re an open source developer has a higher likelihood than GitHub disappearing.
One day, we’re all going back to vendoring dependencies.
Updates mean you have to copy over all the code into your repo, which creates a large diff, and hope you aren't overwriting any local changes someone might have made.
It bloats your repo, both with the actual code, and the large diffs when you update it.
You have to manually track new versions, without something to tell you if new versions are available, or if your version has known security vulnerabilities.
If the dependency has it's own dependencies, you have to vendor those too recursively. And if multiple dependencies have the same transitive dependency, it is up to you to deduplicate them, and make sure you have a version compatible with all dependents.
Because it sucks. But maybe it should suck? It would make us think twice before adding dependencies.
I use dotnet and I never liked seeing dlls and binary files in my diffs. I would argue if we are adding vendor code to our projects, we should demand the FULL source code instead of dlls. Maybe it is already possible with things like x unit. I have never given it much thought... But then that vendoree code has to come from somewhere as well, right? I mean there is something to be said about provenance or something here?
Sorry if this feels like a stream of consciousness because it is ↔
Vendoring dependencies doesn’t necessarily mean you have to include binary objects in your repo. They could be content-addressable artifacts in your org’s private blob store.
I think its a good question. The short answer is: because tooling doesn't default to this or make it easy.
The answer to the question of why THAT is the case - is not so easy to answer. The main benefit I can see with using lock files instead of vendoring is that it saves a lot of storage and diff history from entering your repository. So clones are much faster, backups smaller etc.
I think Go used to work this way (automated vendoring) but it’s the only language I can think of that ever did in terms of standard tooling. It would be instructive to learn why that changed.
And yes I'm aware of the irony of hosting this on GitHub... still figuring out a good workflow for maintaining our OSS on Codeberg and GitHub in parallel fed from the internal forge. The dependency on our own domain is real but we favor it.
> In my opinion, every commerical software development team using Go should be using custom domains for namespacing their internal libraries and packages.
I’d remove “go” from the above, i.e. I think same applies to other stacks.
Even using GitHub domain links in code comments gets problematic long term. Ie when a migration happens and those links start pointing nowhere.
I'm confused by the article, and this comment, treating these URLs as difficult to replace.
Why can't you just search-and-replace? Presumably all of them refer to "GitHub.com" and not much else code will, so I'd think this was an exceptionally easy case.
Even easier for comments, since them being obsolete for a few hours during a migration doesn't exactly break anything.
I've gone through such a migration. Huge Go code base distributed across a lot of git repositories had to be moved to a different git host.
It was horrible. A team of people spent weeks. Different projects depended on different versions of the same internal libraries, we had to create a branch for each depended-on commit and make a version of that commit with the new URLs. The expressed goal was to end up with a system that's "exactly the same" as the old, just on a new host. Changing which version of libraries projects depends on would introduce unnecessary risk.
And the result is a code base where bisects are broken and where it's impossible to build an old version of any of the code without a ton of work.
Even easier than that, you can use the 'replace' statement in your go mod to change where the Go build system will try to pull the dependencies from; you can point to a folder on disk or to another forge, and if you want total control you can indirect everything to your own artifact cache via GOPROXY (which can be something as simple as a static folder of source code).
The "module name is network path" is a convenient convention but not at all some "limitation" of the tooling.
What about anyone else using your code? You can find and replace your own code, but do you have access to all of the code relying on your go library? Is it all yours? Customers? Other developers?
Even in the case where it’s an internal only Lu array, it can be complicated to refactor a library name.
I mean yeah, but other languages don't make it quite so easy to make this mistake. Once your Go project gets to a size where it makes sense for different files to live in different folders (which, given Go's folder-based module system, often happens at just a couple hundred lines of code), the most straightforward way which the tooling nudges you towards means putting GitHub URLs (or URLs to whatever git host website you happen to use) in your source code.
Seems like an error to use a URL. This is perfect for a URN or some other form of URI.
It could be a URN that used the DNS as a back end (though that’s a lot like a URL) so better would be something more abstract with multiple possible resolvers and a signature.
> I'd rather it stayed on Github so it doesn't disappear. Particular for businesses whose priorities might change.
People can delete projects from GitHub. Businesses whose priorities might change may not keep an old project set to public up on GitHub because they won't want people to continue contacting them for support, or they don't want to be responsible for updating security vulnerabilities for projects they are abandoning so they'd rather just pull it offline, etc.
Whether the library you're importing is hosted on GitHub or not, never assume it will be there tomorrow. *Always vendor your dependencies.*
Even better reason: you can later point this domain at an artifact registry. This not only gives you reliability and flexibility, it also secures your software supply chain. You don't need an SBOM or anything fancy to get started, just pull all your artifacts into a central source and improve it over time. Install an artifact registry anywhere you can run a container, use dumb static shared credentials, and start with "proxy mode". Later on you can pin or restrict versions, verify checksums, implement SSO, etc. This is going to become table stakes in the new security landscape.
> That is, if you move your git hosting to GitLab then you have to change your code!
You can also just use "replace github.com/example/example => gitlab.com/example/example" in your go.mod file and everything will keep working. That seems like a very pre-mature optimization for something that doesn't really matter.
cmd+shift+r in goland, push to a branch, open a PR, ask devops to make the old location archived and prevent further writes. After a couple months remove access to the old one and see what breaks.
You know, normal development task, small amount of story points..
You're now arguing that it's easy, which is a completely different argument. The comment I responded to argued that it's unnecessary ("just do the rewrite in go.mod, leave source files unchanged").
Yep that sounds pretty bad. Personally, we'd just eat not being able to build old versions, we archive previous builds, and in the case of a hotfix or whatever we'd use the go.mod rewrite trick. Not sure why you needed the commit/branch setup, did you change the history?
Have libfoo with a bunch of different versions. Authservice depends on libfoo 1.2.0. APIservice depends on libfoo 1.3.0.
Now you want to move from github.com to git.example.org. You change libfoo, authservice and apiservice to use git.example.org, that part is just simple tedious work. But the v1.2.0 and v1.3.0 tags of libfoo are old commits from before the move, so they still reference github.com! Now you need to branch off of the v1.2.0 and v1.3.0 tags of libfoo and do the same change there.
So we have only 3 repositories with only 2 dependencies and we already have to do the search/replace 5 times.
Imagine now that apiservice depends on authservice v2.3.7 just to include some type definitions. Authservice is currently on version 2.4.0 but the types haven't changed so apiservice hasn't upgraded its dependency. Now you need to branch off of authservice v2.3.7 too and do the job there. Oh and authservice v2.3.7 depends on libfoo v1.2.5, so now you need to make a branch off of libfoo v1.2.5 with the search/replace.
3 repositories with a straightforward dependency relationship, 7 search/replace jobs.
What about everyone else using your code? You are going to ask them all to update their code as well? Many of them probably won't realize, and will get stuck on an old version that never gets updated, even to fix bugs and security issues.
I have not worked with Go, so I thought I'd ask: why wouldn't simple find and replace be able to fix this? And why wouldn't a coding agent be able to do this for you quickly?
Thanks for answering my question! So yes, if I intend to use go and study the packages I'd have to rely on beforehand, if they're all hosted on GitHub should make me steer away from Golang. Or at least plan to cache the packages I need locally so CI always have something to work with when building.
Alternatively, could we ourselves build an automatic mirror so there's a redundant supply provider that doesn't depend on a maintainer's choice of git forge
This is great and I think for any company/individual who is going to ensure their domain is registered and maintained this makes a lot of sense. It would be catastrophic, but there is no reason Github couldn't disappear or otherwise change some policies that require moving away from it. The way Go works makes committing these package names tied to Github so much more weighty than simply the place you pull from.
The one thing that I was worried about was returning 301 in the example Nginx config. If you ever wanted to change the url that clients are redirected to then any browsers that visited the old url config would be forced to go to the old config's redirect url. For `go ...` and `curl` it wouldn't matter, but Chrome/Firefox will cache that 301 permanently and break the intended redirect. Not sure if this is really an issue in practice though.
prasadvara | 5 hours ago
SenHeng | 3 hours ago
One day, we’re all going back to vendoring dependencies.
otterley | 3 hours ago
metaltyphoon | 2 hours ago
otterley | an hour ago
thayne | 35 minutes ago
It bloats your repo, both with the actual code, and the large diffs when you update it.
You have to manually track new versions, without something to tell you if new versions are available, or if your version has known security vulnerabilities.
If the dependency has it's own dependencies, you have to vendor those too recursively. And if multiple dependencies have the same transitive dependency, it is up to you to deduplicate them, and make sure you have a version compatible with all dependents.
Etc.
collabs | 2 hours ago
I use dotnet and I never liked seeing dlls and binary files in my diffs. I would argue if we are adding vendor code to our projects, we should demand the FULL source code instead of dlls. Maybe it is already possible with things like x unit. I have never given it much thought... But then that vendoree code has to come from somewhere as well, right? I mean there is something to be said about provenance or something here?
Sorry if this feels like a stream of consciousness because it is ↔
otterley | an hour ago
jeremyjh | 51 minutes ago
The answer to the question of why THAT is the case - is not so easy to answer. The main benefit I can see with using lock files instead of vendoring is that it saves a lot of storage and diff history from entering your repository. So clones are much faster, backups smaller etc.
I think Go used to work this way (automated vendoring) but it’s the only language I can think of that ever did in terms of standard tooling. It would be instructive to learn why that changed.
stackedinserter | 46 minutes ago
deniska | 21 minutes ago
andreashaerter | 3 hours ago
Here's mine: https://github.com/foundata/hugo-theme-govanity (e.g. used at https://golang.foundata.com/ )
And yes I'm aware of the irony of hosting this on GitHub... still figuring out a good workflow for maintaining our OSS on Codeberg and GitHub in parallel fed from the internal forge. The dependency on our own domain is real but we favor it.
thih9 | 2 hours ago
I’d remove “go” from the above, i.e. I think same applies to other stacks.
Even using GitHub domain links in code comments gets problematic long term. Ie when a migration happens and those links start pointing nowhere.
handoflixue | 2 hours ago
Why can't you just search-and-replace? Presumably all of them refer to "GitHub.com" and not much else code will, so I'd think this was an exceptionally easy case.
Even easier for comments, since them being obsolete for a few hours during a migration doesn't exactly break anything.
gravypod | 2 hours ago
mort96 | an hour ago
It was horrible. A team of people spent weeks. Different projects depended on different versions of the same internal libraries, we had to create a branch for each depended-on commit and make a version of that commit with the new URLs. The expressed goal was to end up with a system that's "exactly the same" as the old, just on a new host. Changing which version of libraries projects depends on would introduce unnecessary risk.
And the result is a code base where bisects are broken and where it's impossible to build an old version of any of the code without a ton of work.
lelandbatey | an hour ago
The "module name is network path" is a convenient convention but not at all some "limitation" of the tooling.
mort96 | an hour ago
mbreese | an hour ago
Even in the case where it’s an internal only Lu array, it can be complicated to refactor a library name.
mort96 | an hour ago
gumby | 2 hours ago
It could be a URN that used the DNS as a back end (though that’s a lot like a URL) so better would be something more abstract with multiple possible resolvers and a signature.
skybrian | an hour ago
racingmars | an hour ago
People can delete projects from GitHub. Businesses whose priorities might change may not keep an old project set to public up on GitHub because they won't want people to continue contacting them for support, or they don't want to be responsible for updating security vulnerabilities for projects they are abandoning so they'd rather just pull it offline, etc.
Whether the library you're importing is hosted on GitHub or not, never assume it will be there tomorrow. *Always vendor your dependencies.*
skybrian | an hour ago
bionsystem | an hour ago
0xbadcafebee | an hour ago
prasadvara | an hour ago
-- Curious how this avoids SBOM need?
dewey | an hour ago
You can also just use "replace github.com/example/example => gitlab.com/example/example" in your go.mod file and everything will keep working. That seems like a very pre-mature optimization for something that doesn't really matter.
mort96 | an hour ago
cyberpunk | an hour ago
You know, normal development task, small amount of story points..
mort96 | 58 minutes ago
For my thoughts about why it's not easy, see this comment: https://news.ycombinator.com/item?id=49870363
cyberpunk | 52 minutes ago
mort96 | 35 minutes ago
Now you want to move from github.com to git.example.org. You change libfoo, authservice and apiservice to use git.example.org, that part is just simple tedious work. But the v1.2.0 and v1.3.0 tags of libfoo are old commits from before the move, so they still reference github.com! Now you need to branch off of the v1.2.0 and v1.3.0 tags of libfoo and do the same change there.
So we have only 3 repositories with only 2 dependencies and we already have to do the search/replace 5 times.
Imagine now that apiservice depends on authservice v2.3.7 just to include some type definitions. Authservice is currently on version 2.4.0 but the types haven't changed so apiservice hasn't upgraded its dependency. Now you need to branch off of authservice v2.3.7 too and do the job there. Oh and authservice v2.3.7 depends on libfoo v1.2.5, so now you need to make a branch off of libfoo v1.2.5 with the search/replace.
3 repositories with a straightforward dependency relationship, 7 search/replace jobs.
The numbers get terrifying as you scale this up.
kelnos | 15 minutes ago
jayd16 | 9 minutes ago
mosselman | 58 minutes ago
mort96 | 56 minutes ago
gchamonlive | 55 minutes ago
Alternatively, could we ourselves build an automatic mirror so there's a redundant supply provider that doesn't depend on a maintainer's choice of git forge
https://news.ycombinator.com/item?id=49434625
Meneth | 37 minutes ago
nizarmah | 25 minutes ago
0xCMP | 15 minutes ago
The one thing that I was worried about was returning 301 in the example Nginx config. If you ever wanted to change the url that clients are redirected to then any browsers that visited the old url config would be forced to go to the old config's redirect url. For `go ...` and `curl` it wouldn't matter, but Chrome/Firefox will cache that 301 permanently and break the intended redirect. Not sure if this is really an issue in practice though.