> Emphasis mine. It would indeed be hard to survive without that kind of support from a corporation. A user on HN estimated the yearly cost of this traffic at around 12 million USD/year (according to AWS Cloudfront rates), more than four times the full operating budget of the Python Software Foundation as of 2024.
(As the user in question: three times.)
> Leverage progress in the systems programming ecosystem to create repeatable builds. Turn prebuilt binaries from “sources” into cacheable artifacts that can be deleted and reconstructed at will. Institute a way of creating secondary caches that can start shouldering some of the workload.
This doesn't avoid the need for the wheels to exist and be publicly available. People running CI systems should figure out local caching that actually works, sure. But if you delete that cacheable artifact on the public PyPI website for something like, I don't know, numpy-2.3.2-cp312-cp312-win_arm64.whl, you're going to be re-creating it (and having it downloaded again) constantly. Windows users are just not going to be able to build that locally.
And you know, space usage isn't the problem here — we're talking about a few hard drives' worth of space. The number of downloads is the problem. Outside of CI, I guess that's mostly driven by end users defaulting to the latest version of everything, every time they make a new virtual environment, rather than using whatever's in their package installer's cache. I do know that uv makes the latter a lot easier.
> Windows users are just not going to be able to build that locally.
I think that's being too kind to Linux + Mac users, they are often not generally able to do it either in my experience, especially with Python stuff which tends to need more bleeding edge deps than the local package manager can supply.
(As the user in question: three times.)
> Leverage progress in the systems programming ecosystem to create repeatable builds. Turn prebuilt binaries from “sources” into cacheable artifacts that can be deleted and reconstructed at will. Institute a way of creating secondary caches that can start shouldering some of the workload.
This doesn't avoid the need for the wheels to exist and be publicly available. People running CI systems should figure out local caching that actually works, sure. But if you delete that cacheable artifact on the public PyPI website for something like, I don't know, numpy-2.3.2-cp312-cp312-win_arm64.whl, you're going to be re-creating it (and having it downloaded again) constantly. Windows users are just not going to be able to build that locally.
And you know, space usage isn't the problem here — we're talking about a few hard drives' worth of space. The number of downloads is the problem. Outside of CI, I guess that's mostly driven by end users defaulting to the latest version of everything, every time they make a new virtual environment, rather than using whatever's in their package installer's cache. I do know that uv makes the latter a lot easier.