That's because in the past before stable ABI it meant that when a new Python release came around you could not install binary distributions of extensions for those versions. So people appended more binary releases for newer Python versions to already existing popular versions of of older libraries. That is less and less necessary today, but I do expect that this will be somewhat painful for some packages around new Python releases.
I don’t see the problem with cutting a new release, personally. I can imagine cases where cutting a new release is a lot of process, but that just argues for making it easier to do releases.
There is not a big difference between uploading a new wheel for an existing release or making a new release. The difference is the downstream impact on consumers. If you are pushing a version, you are changing things which invalidates lockfiles for all versions and is non additive.
There’s a subtlety here: PyPI releases are mutable, but only in an additive sense: you could previously add new distributions (sdists/wheels) to an existing release long after that release was made, but you couldn’t modify existing distributions under that release.
That additive property is a logical consequence of how Python has evolved: releases were atomic when they just consisted of sdists, but they can now contain many wheel flavors as well. The consequence of that in turn is that uploads now take N requests instead of 1, and nobody (until now) considered it important to limit the timespan under which those requests could take place.
(This too will change for the better when the “Upload 2.0” API is finished.)
Oh, is that because wheels depend on the platform they're built for? I guess that makes sense.
I'd have expected the upload process to require the user to upload all wheels at once, but I can imagine how that would be difficult if one had different build machines for each platform they're building their wheels for...
Yeah, exactly. It’s an unfortunate consequence of the fact that releases can have multiple wheels, and the independent fact that uploads to PyPI are done one-file-at-a-time. The latter in turn was never designed as such, changing it is just hard for legacy reasons :-)
Oh, is that because wheels depend on the platform they're built for? I guess that makes sense.
If the wheel contains compiled artifacts, yes. You can cross-compile or collate builds into a central spot and then only upload files all at once, but nothing forces you to do that.
Oh, is that because wheels depend on the platform they're built for? I guess that makes sense.
It’s more the part where historically they’re bound to a specific version of cpython, so if you released a wheel, then there was a new cpython release, you might want to fill that in (for every platform you supported).
These days it’s less necessary thanks to the limited API, and the ffi alternatives if you’re willing to handle more raw FFI glue, but if you really want to interact with the interpreter’s guts, or squeeze every last drop of performance you can, it can still be a thing.
Indeed, don't know if other package index (python and other languages npm for example) have this strange behavior of mutable release. That can lead to security risk but also instability as 2 consecutive pip install (or whatever package manager...) can lead to 2 package with the same version number but different package...
aaronm04 | a day ago
Good, but I'm surprised this hasn't been policy for years...
realkc | a day ago
Personally, I'm surprised PyPI releases are mutable in the first place...
mitsuhiko | 23 hours ago
That's because in the past before stable ABI it meant that when a new Python release came around you could not install binary distributions of extensions for those versions. So people appended more binary releases for newer Python versions to already existing popular versions of of older libraries. That is less and less necessary today, but I do expect that this will be somewhat painful for some packages around new Python releases.
[OP] ngoldbaum | 9 hours ago
I don’t see the problem with cutting a new release, personally. I can imagine cases where cutting a new release is a lot of process, but that just argues for making it easier to do releases.
mitsuhiko | 8 hours ago
There is not a big difference between uploading a new wheel for an existing release or making a new release. The difference is the downstream impact on consumers. If you are pushing a version, you are changing things which invalidates lockfiles for all versions and is non additive.
yossarian | 14 hours ago
There’s a subtlety here: PyPI releases are mutable, but only in an additive sense: you could previously add new distributions (sdists/wheels) to an existing release long after that release was made, but you couldn’t modify existing distributions under that release.
That additive property is a logical consequence of how Python has evolved: releases were atomic when they just consisted of sdists, but they can now contain many wheel flavors as well. The consequence of that in turn is that uploads now take N requests instead of 1, and nobody (until now) considered it important to limit the timespan under which those requests could take place.
(This too will change for the better when the “Upload 2.0” API is finished.)
realkc | 10 hours ago
Oh, is that because wheels depend on the platform they're built for? I guess that makes sense.
I'd have expected the upload process to require the user to upload all wheels at once, but I can imagine how that would be difficult if one had different build machines for each platform they're building their wheels for...
yossarian | 7 hours ago
Yeah, exactly. It’s an unfortunate consequence of the fact that releases can have multiple wheels, and the independent fact that uploads to PyPI are done one-file-at-a-time. The latter in turn was never designed as such, changing it is just hard for legacy reasons :-)
[OP] ngoldbaum | 9 hours ago
If the wheel contains compiled artifacts, yes. You can cross-compile or collate builds into a central spot and then only upload files all at once, but nothing forces you to do that.
masklinn | 9 hours ago
It’s more the part where historically they’re bound to a specific version of cpython, so if you released a wheel, then there was a new cpython release, you might want to fill that in (for every platform you supported).
These days it’s less necessary thanks to the limited API, and the ffi alternatives if you’re willing to handle more raw FFI glue, but if you really want to interact with the interpreter’s guts, or squeeze every last drop of performance you can, it can still be a thing.
bport | 23 hours ago
Indeed, don't know if other package index (python and other languages npm for example) have this strange behavior of mutable release. That can lead to security risk but also instability as 2 consecutive pip install (or whatever package manager...) can lead to 2 package with the same version number but different package...