crates.io: development update

57 points by ehamberg 21 hours ago on lobsters | 11 comments

[OP] ehamberg | 21 hours ago

Happy to see there is work underway to detangle crates.io accounts from Github!

hawski | 16 hours ago

How times have changed. In the past I liked the ability to login with Github account in places (not that I was crazy about it). Today I signed up for Tailscale and the Github option was there, but was far less appealing for me. On Tailscale there is an option to use an OIDC identity provider. I used Codeberg and WebFinger on my own domain after some reading.

Maybe it is a common knowledge, but for the curious as it was not at all obvious for me and I learned this all today (it would be nice to have spoiler/details-summary support):

If logging in with an e-mail (example@example.com) on your own domain you put a file at https://example.com/.well-known/webfinger:

{ "subject": "acct:example@example.com",
  "links": [{
      "rel": "http://openid.net/specs/connect/1.0/issuer",
      "href": "https://codeberg.org" }]}

In Codeberg settings create a new OAuth2 application with a redirect URI: https://login.tailscale.com/a/oauth_response and then you get Client ID and Client Secret to give to Tailscale to authorize (?) it.

I guess it will be like that in the future on crates.io as well.

winter | 14 hours ago

I’m kind of surprised this works — does Tailscale ask for the client ID/secret before redirecting you to Codeberg?

hawski | 14 hours ago

I did this once and learned along the process, so I may not remember it fully. I did start sign up on Tailscale and then it wanted Webfinger. So that's what I did first. Then Tailscale asked for client ID/secret. So I read a bit and then did the thing on Codeberg to obtain those (not a redirect yet). Then back to Tailscale and give it generated client ID/secret. Then it redirected to Codeberg which asked if I want to authorize it. Then back to Tailscale.

So reading https://example.com/.well-known/webfinger is a way to harvest email addresses to spam. I guess one could use a dedicated email address and whitelist only the services where it is used.

BD103 | 17 hours ago

Wow, I didn’t realize how much crates.io has improved recently! I usually go to lib.rs by default, but that may change soon.

My personal favorite change is Ferris :)

mitsuhiko | 15 hours ago

Wonder how much of this can be attributed to using agents there now.

chrismorgan | 17 hours ago

The Ferris change doesn’t make much physical sense: the white is a specular highlight on a black eye. Unless… your mouse cursor is the illumination source? But then it’s not eyes following the cursor.

pflanze | 14 hours ago

This makes it much easier to audit your dependencies

Here's what I've been doing:

  • Once, run cd ~; ln -s .cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ cargosrc to make it easy to get to the files downloaded by cargo

Then, for each dependency that I want to audit:

  • clone its Git repository
  • check out the version that I want to audit
  • delete all locally checked out files, without committing
  • use a script to send SIGSTOP to all rust-analyzer instances; edit Cargo.toml in my project where I want to use the dependency, add it, run cargo fetch
  • cd ~/cargosrc/$dependency-$version/; cp -a . $path_to_git_clone
  • In the Git clone, add and commit those files (git add .; git commit -m crate)
  • Now look through the differences using a Git history viewer.

When happy, I copy the dependency/version/hash triple from Cargo.lock to a trusted database via another script (so that I don't re-check the same dependency/version again in the future).

Which may look more involved, but I think given that I have the workflow down and largely automated, this will remain more effective than looking at the files in the web UI. I would totally welcome if cargo gained some features to help with this on the command line. (Are there any cargo plugins in this area? I know about cargo crev and similar (although not used much), I rather mean functionality to help the review process itself.)

bakkot | 11 hours ago

The frontend then fetches only the manifest and loads each file on demand with an HTTP range request.

Nice. Looks like the implementation is here. It uses the browser-native DecompressionStream.

sanxiyn | 10 hours ago

TIL DecompressionStream. Thanks!