pixi doesn't install specified commit of pypi git dependency

[beenje]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

This is with pixi 0.35.0.

Considering the following pixi.toml:

[project]
channels = ["conda-forge"]
name = "test-pixi"
platforms = ["osx-arm64", "linux-64"]

[tasks]
check = "grep orphan .pixi/envs/default/lib/python3.12/site-packages/sphinx_tags/__init__.py"

[dependencies]
python = "3.12.*"
pip = ">=24.3.1,<25"
sphinx = "*"

[pypi-dependencies]
# sphinx-tags = { git = "https://github.com/beenje/sphinx-tags.git", rev = "490f40f" }
sphinx-tags = "==0.4"

If I run pixi run check, I get the expected output (only one line with orphan).

$ pixi run check
✨ Pixi task (check): grep orphan .pixi/envs/default/lib/python3.12/site-packages/sphinx_tags/__init__.py
        content.append(":orphan:")

I now replace the sphinx-tags version with a specific commit (the line orphan: true was added in this commit).

[pypi-dependencies]
sphinx-tags = { git = "https://github.com/beenje/sphinx-tags.git", rev = "490f40f" }

$ rm -rf .pixi/ pixi.lock
$ pixi run check
✨ Pixi task (check): grep orphan .pixi/envs/default/lib/python3.12/site-packages/sphinx_tags/__init__.py
        content.append(":orphan:")

The old version is still installed despite the proper version in pixi.lock:

$ grep sphinx-tags pixi.lock
      - pypi: git+https://github.com/beenje/sphinx-tags.git@490f40f0042792ca410c249e195f29c96e33b25b
      - pypi: git+https://github.com/beenje/sphinx-tags.git@490f40f0042792ca410c249e195f29c96e33b25b
  name: sphinx-tags
  url: git+https://github.com/beenje/sphinx-tags.git@490f40f0042792ca410c249e195f29c96e33b25b

If I remove rattler cache, I get the proper version installed:

$ rm -rf .pixi /root/.cache/rattler/cache
$ pixi run check
✨ Pixi task (check): grep orphan .pixi/envs/default/lib/python3.12/site-packages/sphinx_tags/__init__.py
        content.append("orphan: true")
        content.append(":orphan:")

Issue description

I believe the issue is the same as what was reported in #2358, but closed by the author who couldn't reproduce.

There seems to be a cache issue preventing the specified git version to be installed, when another version was installed previously.

Expected behavior

Removing the cache shouldn't be needed to install a specific git version of a package.

[ruben-arts]

Thank you for this great reproducer. It even works on my windows machine. We do need to look into this. It also doesn't work with an earlier version.

[ppinchuk]

I am running into a similar (likely related issue) with editable local installs.

With Pixi version: 0.39.5, starting with the following pixi.toml:

[project]
channels = ["conda-forge"]
name = "pixi_test"
platforms = ["linux-64"]

[tasks]
check = "python -c 'import yaml; print(yaml.__file__)'"

[dependencies]
python = ">=3.9,<3.14"

[pypi-dependencies]
pyyaml = "==6.0.2,<7"

Running pixi r check, we see that the file is installed under .pixi/envs/default as expect (i.e. pulled from PyPi):

❯ pixi r check         
✨ Pixi task (check): python -c 'import yaml; print(yaml.__file__)'
/home/user/pixi_test/.pixi/envs/default/lib/python3.13/site-packages/yaml/__init__.py

Now I want to use a local version of PyYaml (e.g. I am developing a new feature). I grab the code using
git clone --branch 6.0.2 [email protected]:yaml/pyyaml.git and update the pixi.toml to be:

[project]
channels = ["conda-forge"]
name = "pixi_test"
platforms = ["linux-64"]

[tasks]
check = "python -c 'import yaml; print(yaml.__file__)'"

[dependencies]
python = ">=3.9,<3.14"

[pypi-dependencies]
pyyaml = { path = "pyyaml", editable=true }

However, running pixi r check, we see that the file is still under .pixi/envs/default, and not in /home/user/pixi_test/pyyaml as expected:

❯ pixi r check         
✨ Pixi task (check): python -c 'import yaml; print(yaml.__file__)'
/home/user/pixi_test/.pixi/envs/default/lib/python3.13/site-packages/yaml/__init__.py

We can further confirm that the editable install is not working by making modifications in the local pyyaml code without having any effect on the environment. The lockfile also seems to point to the local code

- pypi: pyyaml
  name: pyyaml
  version: 6.0.2
  sha256: a6b524f6b5cfd05d04d2111a770576c94511ef77f453e28cb5c099068bb63f23
  requires_python: '>=3.8'
  editable: true

but is being ignored. Removing pixi.lock and .pixi has no effect - the file is still under .pixi/envs/default after re-building.

The only thing that works is rm -r ~/.cache/rattler/cache .pixi pixi.lock. After clearing the cache, we get the expected result:

❯ pixi r check                                
✨ Pixi task (check): python -c 'import yaml; print(yaml.__file__)'
/home/user/pixi_test/pyyaml/lib/yaml/__init__.py

Additional context

This seems to be exclusively a problem with [pypi-dependencies]. If we repeat the entire exercise with conda deps (i.e. start with

[project]
channels = ["conda-forge"]
name = "pixi_test"
platforms = ["linux-64"]

[tasks]
check = "python -c 'import yaml; print(yaml.__file__)'"

[dependencies]
python = ">=3.9,<3.14"
pyyaml = "==6.0.2,<7"

and then switch to

[project]
channels = ["conda-forge"]
name = "pixi_test"
platforms = ["linux-64"]

[tasks]
check = "python -c 'import yaml; print(yaml.__file__)'"

[dependencies]
python = ">=3.9,<3.14"

[pypi-dependencies]
pyyaml = { path = "pyyaml", editable=true }

), then everything works exactly as expected.

[ruben-arts]

@ppinchuk Thanks for the great report!

I think I have a clue where this is going wrong. We have a plan to improve the git logic of pixi soon. This probably will also improve this example.

[ppinchuk]

Thanks!!

[ruben-arts]

Sorry for the delayed response @ppinchuk, we've found the issue but weren't able to fix it yet. The current code path is not setup to properly find the source build cache vs the remote cache. Thus the issue persists. We did fix it partly as the code now knows that it "should" reinstall: #3131

[ppinchuk]

No worries, thanks for continuing to look into it!

[ruben-arts]

As this issue isn't related with the actual issue I've opened a new one with your example @ppinchuk: #3151. Closing this issue as the original one was fixed.