Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partial implementation of source groups #13210

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

msarahan
Copy link

@msarahan msarahan commented Feb 7, 2025

I'm submitting this more as a point of discussion than as code that should be carefully reviewed and considered for inclusion.

PEP 766 left me uneasy with how disruptive it would likely be to pip. I had an idea that I hope might give us the best of both worlds: keeping pip's current behavior, while also adding orthogonal functionality that gives users control over prioritizing some software sources over others.

This code is a partial implementation of that idea, and I wanted to run it past the pip maintainers to see if it is worth developing fully. The idea is that instead of changing the way that sources work, you add the notion of source groups. This is like poetry's repository priority. This PR is an effort to adapt that a little bit, such that:

  • The only pre-defined group is the default group that the existing flags go into
  • The number of groups is user-defined, not limited to pre-existing named groups

Between groups, the behavior is similar to the index priority described in the PEP - that is, similar to uv's "first match" default behavior. Unlike uv, the behavior within a group is pip's "version priority" behavior.

To achieve this, I renamed the existing PackageFinder class to InternalPackageFinder, and wrapped it with a class that is meant to manage multiple InternalPackageFinder instances. Its logic is almost totally pass-through to the "currently selected" InternalPackageFinder. The PackageFinder class iterating over InternalPackageFinder instances is what provides the priority behavior among groups.

Most of the work in this PR so far has been refactoring the CLI options to move them up, so that existing classes could be reused with options that may have different values across different source groups (or IndexGroups - I am not settled on the best name here).

Expected user-facing changes:

  • Existing configuration will continue to work exactly as-is
  • Users wishing to utilize the priority feature would need to add new configuration

Outstanding questions:

  • Is this scheme reasonable and worth more time to develop?
  • What should configuration look like?

The pip.conf file being flat makes this hard. I don't like the idea of having a separate configuration file. The idea I'm mulling right now is to add a flag and pip.conf equivalent that concatenates the different options, like:

--index-group=group_name:index_urls=https://blah/simple,https://more/simple;find_links=...;prefer_binary=true;...

pip.conf would be a little better, but concatenation is still pretty ugly:

index-groups =
    red-hot-chili-peppers:index_urls=https://blah/simple,https://more/simple;find_links=...;prefer_binary=true;...
    oasis:index_urls=https://blah/simple,https://more/simple;find_links=...;prefer_binary=true;...
index_group_priority =
    oasis
    red-hot-chili-peppers

I don't like this idea, but I prefer it to the notion of a new file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant