Pyx represents the server side, not the client side. The analogue in the pre-existing Python world is PyPI.
Many ideas are being added to recent versions of pip that are at least inspired by what uv has done — and many things are possible in uv specifically because of community-wide standards development that also benefits pip. However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original). That has a lot to do with why I'm making PAPER.
For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. For pip to follow suit, it would have to completely redo its caching strategy from the ground up, because right now its cache is designed to save only download effort and not anything else about the installation process. It remembers entire wheels, but finding them in that cache requires knowing the URL from which they were downloaded. Because PyPI organizes the packages in its own database with its own custom URL scheme, pip would have to reach out to PyPI across the Internet in order to figure out where it put its own downloads!
> However, pip has some really gnarly internal infrastructure that prevents it from taking advantage of a lot of uv's good ideas (which in turn are not all original).
FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.
I'm not saying here that pip doesn't have some gnarly internal details, just that the bigger thing holding it back is the lack of maintainer resources.
> For just one example: uv can quickly install previously installed packages by hard-linking a bunch of files from the cache. For pip to follow suit, it would have to completely redo its caching strategy from the ground up, because right now its cache is designed to save only download effort and not anything else about the installation process.
I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy.
That said, I do think there are other good examples to your point, like uv's ability to prefetch package metadata, I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver.
> FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.
Fair enough. I'm sure if someone were paying me a competitive salary to develop my projects, they'd be getting done much faster, too.
> I actually think this isn't a great example, evidenced by the lack of a download or wheel command from uv due to those features not aligning with uv's caching strategy.
I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files.
> uv's ability to prefetch package metadata,
You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays.
> I don't think we're going to be able to implement that in pip any time soon due to probably the need for a complete overhaul of the resolver.
Ugh, yeah. I know the resolution logic has been extracted as a specific package, but it's been challenging trying to figure out how to actually use that in a project that isn't pip.
> I guess you're talking about the fact that uv's cache only stores the unpacked version, rather than the original wheel? I'm planning to keep the wheel around, too. But my point was more that because of this cache structure, pip can't even just grab the wheel from its cache without hitting the Internet, on top of not having a place to put a cache of the unpacked files.
I'm talking about the fact there is no `uv pip download` or `uv pip wheel`.
I'm sure we'll discuss when you add this feature to uv, but I personally find uv's cache already grows too big too fast, so adding more to it makes me concerned.
> You mean, as opposed to obtaining it per version, lazily? Because it does seem like the .metadata file system works pretty well nowadays.
Yeah, one of the way uv speeds up resolving is it pre-downloads .metadata files and checks if their requirements are identical to versions it had already checked, so it can quickly rule them out. It's a clever use of a collector and an advanced resolver.
>FWIW, as a pip maintainer, I don't strongly agree with this statement, I think if pip had the same full time employee resources that uv has enjoyed over the last year that a lot of these issues could be solved.
No. This something people tell each other, that it's a lack of resources, but in reality almost all OSS projects with long standing flaws don't have a resources problem. They have a prioritization problem, where they actively ignore and refuse to work on things that affect users every single day of usage.
There are features in FreeCAD that are straight up just broken that you hit every single day of using FreeCAD. When a simple low cost fix is suggested you get immense pushback, because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance.
Progress happens when you sidestep those who block progress. That's what realthunder was doing and that's what Astral is doing with uv. No more bullshit excuses.
This is what forks are for. Alternately, money would probably help. Otherwise it's a bit rude to impose your priorities on others.
Perhaps some larger projects lose track of what they intended to prioritize. This is also a resources problem. There's nobody available to do management (and probably also not a social structure internally that would allow for it).
> because of how "impure" the fix is despite being entirely cosmetic, also being reversible and having no long term impact on maintenance.
The developers will always be in a better position than end users to assess the long-term maintenance impact.
> The developers will always be in a better position than end users to assess the long-term maintenance impact.
At best developers may be in a better position to assess long-term consequences. The history of major software projects is also the history of many projects making technical decisions that look good in the short-term but which turn out to impede actually achieving the desired features/performance/etc, and thus lead to major refactors/rewrites.
imtringued is right though. Obstinate maintainership is a real failure mode of open source (and other) projects and it's often true that patch rejections involving phrases like "maintenance burden" are often pretextual. It's a shame to have to resort to creative destruction to improve things. Forks have a low success probability.
> Obstinate maintainership is a real failure mode of open source
I agree, but I also think sometimes that people think it's obstinate maintainership and it's not. For example, large PRs are hard to merge from a contributor who is fresh to the project because there is so much careful review that needs to be done, it's very resource intensive.
That said, one of my goals of becoming a maintainer has to be to try and making submitting a PR more friendly. Feel free to submit a PR to pip, I will be happy to help you get over any teething issues.
> No. This something people tell each other, that it's a lack of resources,
Well uv, as an OSS project, has come along with around 2 orders of magnitude more manpower than pip and has solved a lot of problems
> but in reality almost all OSS projects with long standing flaws don't have a resources problem. They have a prioritization problem, where they actively ignore and refuse to work on things that affect users every single day of usage.
I wasn't making a statement about OSS projects in general, but I agree that pip has prioritization problems but I would argue that it stems from the lack of resources.
The people who do donate their spare time are in fact only likely to donate that time on things that interest them. If there were resources to have someone act like a project manager, and other to follow their lead, then the prioritization problems could be fixed, but there aren't those resources.
> No more bullshit excuses
Then contribute your full time hours to pip, or other OSS that needs fixing, rather than arm chair commenting on hacker news.
Good to know. "which in turn are not all original" may have been an understatement. I don't know a lot of these details, because my interest in these things is more recent, and not so much driven by the complex multi-language projects.
This doesn't generalize: you could have said the same thing about pip versus easy_install, but pip clearly has worthwhile improvements over easy_install that were never merged back into the latter.
Pip is broken and has been for years, they're uninterested in fixing the search. Or even removing the search or replacing it with a message/link to the package index.
imo, if pip's preference is to ship broken functionality, then what is/is not shipped with pip is not meaningful.
This is not a charitable interpretation. The more charitable read is that fixing search is non-trivial and has interlocking considerations that go beyond what pip's volunteer maintainers reasonably want to or can pick up.
(And for the record: it isn't their fault at all. `pip search` doesn't work because PyPI removed the search API. PyPI removed that API for very good reasons[1].)
That was 7 years ago. If it's not coming back, the CLI should make that clear, instead of giving a temporary "cannot connect" message that implies it could work, if you wait a minute and try again.
It was three years ago; 2018 is when they considered removing the command, not when the search API was actually removed from PyPI.
And this is part of the interlocking considerations I mentioned: there are private indices that supply the XML-RPC API, and breaking them doesn't seem justifiable[1].
Does that seem like a real solution to you? That it's ok to represent a never-functional operation as one that might maybe work? ...because it could work of you jump through a bunch of undocumented hoops?
It's so wild to me that so many people are apparently against making a user-friendly update. The whole thing seems very against pep8 (its surprising, complicated, non-specific, etc)
I don't know what to tell you; I just gave an example of it being functional for a subset of users, who don't deserve to be broken just because it's non-functional on PyPI.
Nobody wants anything to be user-unfriendly. You're taking a very small view into Python packaging and extending it to motives, when resources (not even financial ones) are the primary challenge.
Is code to detect when the user is not in that subset and say that it doesn't work really really hard for some non-obvious reason? If the default case for the vast majority of users doesn't work, it doesn't seem like printing a more useful error message to them should be that hard.
> it doesn't seem like printing a more useful error message to them should be that hard.
I think the existing error message is useful:
$ pip search foo
ERROR: XMLRPC request failed [code: -32500]
RuntimeError: PyPI no longer supports 'pip search' (or XML-RPC search). Please use https://pypi.org/search (via a browser) instead. See https://warehouse.pypa.io/api-reference/xml-rpc.html#deprecated-methods for more information.
It says (1) what failed, (2) why it failed, and (3) links to a replacement, and (4) links to a deprecation explanation. That last link could maybe then link back to the pip issue or include some more context, but it's a far cry from being not helpful.
That is a nicer message than I remember. I don't have an issue with that. It let's you know it doesn't work, that it will never work, and provides links to an alternative and more information.
It either used to do fewer of those, or my memory is Swiss cheese, but I remember needing to search for the why's and alternatives.
There was a period when the error was a stack trace. But that was pretty brief, and happened before PyPI moved more "officially" towards not supporting the search API.
I think part of the confusion here (which is reasonable IMO) is that pip has a huge version tail, and so people report error messages and behaviors that have been fixed for years. This is one of the challenges that I think gets under-appreciated about pip's maintenance.
Your comment gave me an flashback to when I started programming and drag and dropped downloaded python packages into the packages folder instead of installing them.
I'm assuming by pip you mean pypi (the package registry) - I think you're making the mistake of thinking it has a lot more resources than it does, because of how much it punches above its weight.
Pypi is powered by warehouse which has around 3 developers maintaining it[0]. They're doing an awesome job, but the funding and resource available to them are probably substantially less than Astral could have with a paid offering.
