partial implementation of source groups #13210
Draft
+155
−21
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm submitting this more as a point of discussion than as code that should be carefully reviewed and considered for inclusion.
PEP 766 left me uneasy with how disruptive it would likely be to pip. I had an idea that I hope might give us the best of both worlds: keeping pip's current behavior, while also adding orthogonal functionality that gives users control over prioritizing some software sources over others.
This code is a partial implementation of that idea, and I wanted to run it past the pip maintainers to see if it is worth developing fully. The idea is that instead of changing the way that sources work, you add the notion of source groups. This is like poetry's repository priority. This PR is an effort to adapt that a little bit, such that:
Between groups, the behavior is similar to the index priority described in the PEP - that is, similar to uv's "first match" default behavior. Unlike uv, the behavior within a group is pip's "version priority" behavior.
To achieve this, I renamed the existing PackageFinder class to InternalPackageFinder, and wrapped it with a class that is meant to manage multiple InternalPackageFinder instances. Its logic is almost totally pass-through to the "currently selected" InternalPackageFinder. The PackageFinder class iterating over InternalPackageFinder instances is what provides the priority behavior among groups.
Most of the work in this PR so far has been refactoring the CLI options to move them up, so that existing classes could be reused with options that may have different values across different source groups (or IndexGroups - I am not settled on the best name here).
Expected user-facing changes:
Outstanding questions:
The pip.conf file being flat makes this hard. I don't like the idea of having a separate configuration file. The idea I'm mulling right now is to add a flag and pip.conf equivalent that concatenates the different options, like:
--index-group=group_name:index_urls=https://blah/simple,https://more/simple;find_links=...;prefer_binary=true;...
pip.conf would be a little better, but concatenation is still pretty ugly:
I don't like this idea, but I prefer it to the notion of a new file.