There is an implicit trust issue though. I know that I can verify what code is being served from github because I trust the integrity of github's hosted service. I don't have the same trust towards domains I don't recognize. What if a malicious domain shows one thing when I visit it with a browser and serves something different when go fetches dependencies?
To be exact, the Go module proxy makes sure content doesn't mutate over time, and does not verify that what is shown in the web UI matches what is fetched by git (which seems to be what confusedcyborg is getting at).
I think the argument isn't that a malicious site would serve something different to the Go tooling, but that there would be some human-usable UI purporting to show the source code for audit purposes, but when the Go tooling pulls the repo, different code is delivered. Obviously you can still check once the code makes it to your machine, of course, but it's more of a hassle.
A malicious module author can do that with GitHub-hosted modules, by rewriting tags or force-pushing branches after the module proxy has cached the malicious version.
Sure, GitHub doesn't let you serve different content for browsers than Go tooling. But the issue of trust is really of the author/organization. We have plenty of examples of compromised source/packages on popular hosting infrastructure. Pulling a module hosted under someone's pseudonymous GitHub handle isn't really any more trustworthy than from their own domain. The Go module proxy also mitigates the specific attack by having a single signature for a published version of a module.
Ok, this is a much better solution, especially for an individual (rather than a corporation). While you still need to pay for domain name hosting in perpetuity, you don't need to worry about the upkeep (and security) of an actual server just for a redirect.
if you move your git hosting to GitLab then you have to change your code! Otherwise, you’ll be fetching the old verison.
I hosted a few Go modules on my own domain, but I regret it. The effort to maintain them has been more costly than a host move that hasn't happened.
Abstractions come at a cost and it depends if that cost is greater than the effort to move hosts in the future. If I move hosts in the future I expect a greater cost will be migrating all the non-code artifacts: releases, ci, issues, discussions, etc.
There's also the question of what's better for importers. I'd prefer the import be a code host I trust.
That's a tough one: I am conceptually on board with the thesis, and yet folks that do this trickery always add one extra step in my "sure, but where is the source?" quest. I have to remember the ?go-get=1 syntax, use view-source, and then mentally substitute tho variables into the meta declarations
That silly combined with the (IMHO) absolute horror show of replace in go.mod are things I really could have done without in golang
I think putting URLs into the package names was a mistake. If I was designing this, I would have had a local registry of package => URL, which in addition to making it easy to locally replace packages with forks or hacked up versions for testing, would also prevent you from being tied to a specific service provider's name.
The user should be responsible for how their local namespace maps to global resources. You'd have expected the Plan 9 people designing go to get this right.
No. I mean that every package name is mapped this way in go.mod, rather than using a URL in the source. The point is that the naming becomes a local property, not a global one -- which also opens the door for doing neat things, like naming by signature, or other non-human-readable global identifiers.
Except there are urls in the source, which will break if a domain goes down. You could, in theory, do a local remapping, but the patterns that tools nudge you into matter.
If all dependencies are vendored, this is less critical, though. Then projects can just use the vendored dependencies when moving away from ie. GitHub.
confusedcyborg | a day ago
There is an implicit trust issue though. I know that I can verify what code is being served from github because I trust the integrity of github's hosted service. I don't have the same trust towards domains I don't recognize. What if a malicious domain shows one thing when I visit it with a browser and serves something different when go fetches dependencies?
twp | a day ago
The Go module proxy detects these kind of attacks. If the server doesn't deliver the exact same bytes every time then you'll get an error.
kiyurica | 23 hours ago
To be exact, the Go module proxy makes sure content doesn't mutate over time, and does not verify that what is shown in the web UI matches what is fetched by git (which seems to be what confusedcyborg is getting at).
georgelesica | 23 hours ago
I think the argument isn't that a malicious site would serve something different to the Go tooling, but that there would be some human-usable UI purporting to show the source code for audit purposes, but when the Go tooling pulls the repo, different code is delivered. Obviously you can still check once the code makes it to your machine, of course, but it's more of a hassle.
terinjokes | 20 hours ago
You can use https://pkg.geomys.dev/ to view the source of modules, which uses HTTP range requests against the Go Module Proxy's zip files.
icholy | 8 hours ago
I'm surprised that pkg.go.dev doesn't have this built-in
agwa | 11 hours ago
A malicious module author can do that with GitHub-hosted modules, by rewriting tags or force-pushing branches after the module proxy has cached the malicious version.
rsesek | 20 hours ago
Sure, GitHub doesn't let you serve different content for browsers than Go tooling. But the issue of trust is really of the author/organization. We have plenty of examples of compromised source/packages on popular hosting infrastructure. Pulling a module hosted under someone's pseudonymous GitHub handle isn't really any more trustworthy than from their own domain. The Go module proxy also mitigates the specific attack by having a single signature for a published version of a module.
mxey | 14 hours ago
Git tags are mutable, so their tree might be different from what it was when the module proxy fetched and cached it.
oliverpool | 14 hours ago
My own take: a static doc generator which informs the go tool of my current forge https://code.pfad.fr/vanitydoc/
StevenAndrewCoffman | 10 hours ago
Ok, this is a much better solution, especially for an individual (rather than a corporation). While you still need to pay for domain name hosting in perpetuity, you don't need to worry about the upkeep (and security) of an actual server just for a redirect.
spc476 | 5 hours ago
You still have to worry about security as someone could compromise the server and change the redirect.
leighmcculloch | 17 hours ago
I hosted a few Go modules on my own domain, but I regret it. The effort to maintain them has been more costly than a host move that hasn't happened.
Abstractions come at a cost and it depends if that cost is greater than the effort to move hosts in the future. If I move hosts in the future I expect a greater cost will be migrating all the non-code artifacts: releases, ci, issues, discussions, etc.
There's also the question of what's better for importers. I'd prefer the import be a code host I trust.
oliverpool | 14 hours ago
Did you host the whole code, or just a redirection ? (As this article suggests)
agwa | 11 hours ago
Why does it matter to you as an importer?
mdaniel | 23 hours ago
That's a tough one: I am conceptually on board with the thesis, and yet folks that do this trickery always add one extra step in my "sure, but where is the source?" quest. I have to remember the ?go-get=1 syntax, use view-source, and then mentally substitute tho variables into the meta declarations
That silly combined with the (IMHO) absolute horror show of
replaceingo.modare things I really could have done without in golangmro | 14 hours ago
don't rely on a party, that doesn't (or cannot) care about you.
Use addresses under your authority. Redirects are fine.
orib | 3 hours ago
I think putting URLs into the package names was a mistake. If I was designing this, I would have had a local registry of package => URL, which in addition to making it easy to locally replace packages with forks or hacked up versions for testing, would also prevent you from being tied to a specific service provider's name.
The user should be responsible for how their local namespace maps to global resources. You'd have expected the Plan 9 people designing go to get this right.
mdaniel | 2 hours ago
Which is what the
replacekeyword does in go.mod: both for forks as well as the infinitely handy local directory (i.e. a file URL I guess)Unless you mean a global registry, which I think
GOPROXY=may accomplish but I've not yet tried thatorib | 2 hours ago
No. I mean that every package name is mapped this way in go.mod, rather than using a URL in the source. The point is that the naming becomes a local property, not a global one -- which also opens the door for doing neat things, like naming by signature, or other non-human-readable global identifiers.
see also https://en.wikipedia.org/wiki/Zooko%27s_triangle
dlisboa | 2 hours ago
What you're describing is exactly how the Go toolchain already works today. It is as easy as you're alluding to.
orib | an hour ago
Except there are urls in the source, which will break if a domain goes down. You could, in theory, do a local remapping, but the patterns that tools nudge you into matter.
leighmcculloch | 18 hours ago
xyproto | 12 hours ago
If all dependencies are vendored, this is less critical, though. Then projects can just use the vendored dependencies when moving away from ie. GitHub.