Inspect system to determine if musl toolchains should be used #4242
Comments
zanieb
added
enhancement
|
Hmm I'm trying to figure it out by This is the closest I can do (you could also look at what files are in Comparing to regular debian # grep -F musl "$(which ldd)"
# grep -F libc "$(which ldd)"
TEXTDOMAIN=libcNaturally you don't need |
|
I wonder if there are other Rust projects that get this right, could we look at how they handle it? |
|
🤷♂️ this rust-lang/rust#71564 links here https://github.com/rust-lang/rust/blob/master/src/ci/docker/scripts/musl-toolchain.sh which could just as easily be checked by file presence | absence in the My little |
|
Impressive sleuthing |
|
We had a relatively comprehensive way of doing this before: https://github.com/astral-sh/uv/blob/e0ac5b4e84d2544c915b83bfb50776b468cdb38d/crates/platform-host/src/linux.rs I'm not sure there are problems with it — mostly I think the plan was to make it simpler since we don't actually need to parse version numbers. |
|
Why was it removed? I can't remember. |
|
The comment at #4160 (comment) provides important context on the current state of things. It was removed in #2381 when we moved to relying on Python's platform detection — we of course can't do this to procure Python itself, but that was before we supported downloads. |
|
Basically, we should totally add support back and we can probably mostly copy the old implementation from the |
|
Happy to tackle that. What error were you getting? |
|
So there's two parts
For (1) we'll probably want to simplify the code some, if feasible. It'd be applied here. |
|
@konstin could give far more context on (2) |
|
Hmm not sure we need all that proper parsing of ELF headers and such. Is there a special reason you need to know which specific version of a libc is available? - Otherwise a simple function that reads a chunk of bytes from a file, searches for magic string, read next chunk, if present return E.g., I sent the MonetDB guys this little C change SamuelMarks/MonetDB@4496ba9#diff-245eb970cc6ec49fc0821f3bf591bcbeb0667b94e30e6296aa24de18c80d35d0R85 but that would require a whole toolchain whereas literally checking filesystem locations or the bytes your So we could I guess parse the header file musl https://github.com/rofl0r/musl/blob/master/include/features.h glibc https://sourceware.org/git/?p=glibc.git;a=blob;f=include/features.h;h=093de6f44cd168fe1e8727851af1baf666ae57fc;hb=HEAD and infer from there. Or a simple line count / file size 😆 |
|
The major problem is that python-build-standalone currently has statically linked musl builds, which means they can't load any extension modules. We need to fix this before we can ship this toolchain to users. uv itself may be linked to glibc or musl, which shouldn't make a difference (we could ship statically musl linked uv for all linuxes - uv itself should look dependency-less to the user). For downloading python, we need to determine if should prefer musl or glibc, some heuristic like the We don't need to determine glibc or musl version to determine the python download, but it would be helpful to enable things like #6212. |
|
@konstin -- Personally it seems worthwhile to get this right in uv even if the musl builds are limited. |
|
Yeah if the current ones (libc) can't be used at all, I think it's an important improvement to download the correct musl distribution.
Yeah I don't think we need to know specific versions anymore — maybe it'd be nice have later but if you're willing to contribute a "simple" version I'd love that. |
|
I believe we should detect musl targets, but not download musl python-build-standalone. Without being able to load extension-modules, this interpreter will fail on any non-trivial project. Using If we treat static python as a If i instead hack in installation, i get an equally unhelpful runtime error (my native host with the musl python-build-standalone hacked in): I agree that the current situation of downloading glibc python on musl platforms such as alpine is also untenable. I suggest restoring It would obviously be great to fix the musl builds to be dynamically linked, but i lack the build system to do that kind of change. |
|
Hacking around with whatever is in Alpine: Looks like it works just fine with the packages you referenced. Ok so let's look: Two deps:
Now let's consider how to solve this problem. Having a bunch of linkable shared libraries doesn't seem horrible, and it can still be basically free-standing by putting each installation in its own self-contained directory (I think Nix does this). Shouldn't be that hard, just a bit of grunt work, there's only like 17 archives to consider here and we can use alpine's mirrors. We don't need to use their package manager even, their archive format is pretty simple can stick to Rust at the source-code rather than process-execution level if you like. |
FWIW, Alpine is switching to a custom package format with the (upcoming, still TBA) release of apk-tools v3; the better option here would be to just compile dynamic musl builds under a musl-based distribution ( whether it would be done by pbs or astral ) and copy the dynamic libraries in the same version it was compiled against, rather than fetch the dynamic libraries from distro packages manually ( that is, if i'm even understanding this correctly ) |
one little nitpick for when that happens: using |
|
also, i was able to build somewhat cursed CPython environments with dynamic linking by... extracting Docker containers: build-standalone-musl-python.sh#!/bin/sh
set -eux
alpine_version="$1"
arch="$2"
case "$arch" in
x86) docker_arch="i386" ;;
x86_64) docker_arch="library" ;;
aarch64) docker_arch="arm64v8" ;;
*) echo "unknown arch: $arch" && exit 1 ;;
esac
container_id="$(docker create docker.io/$docker_arch/alpine:"$alpine_version" apk add --no-cache python3)"
docker start -ai "$container_id"
docker export "$container_id" | tar x usr lib
docker rm "$container_id"
rm -r lib/apk usr/share usr/sbin
mv lib/* usr/lib/
rmdir lib
patchelf --add-rpath '$ORIGIN/../lib' usr/bin/python3.*
python_version="$(./usr/lib/ld-musl-*.so.1 ./usr/bin/python3 -V | cut -d' ' -f2)"
archive_name="cpython-$python_version-linux-$arch-musl"
mv usr "$archive_name"
tar czf "$archive_name.tar.gz" "$archive_name"
rm -rf "$archive_name"which then seems to work just fine with uv ( after patching $ uv python list --only-installed
cpython-3.12.5-linux-x86_64-musl /usr/bin/python3.12
cpython-3.12.5-linux-x86_64-musl /usr/bin/python3 -> python3.12
cpython-3.12.5-linux-x86_64-musl /usr/bin/python -> python3
cpython-3.10.14-linux-x86_64-musl /home/patrycja/.local/share/uv/python/cpython-3.10.2-linux-x86_64-musl/bin/python3 -> python3.10
cpython-3.9.17-linux-x86_64-musl /home/patrycja/.local/share/uv/python/cpython-3.9.17-linux-x86_64-musl/bin/python3 -> python3.9
$ cat test.py
# /// script
# dependencies = [
# "numpy"
# ]
# ///
import sys
import numpy as np
a = np.arange(15).reshape(3, 5)
print(a)
print(sys.version_info)
$ uv run --python-preference=system -- test.py
Reading inline script metadata from: test.py
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]]
sys.version_info(major=3, minor=12, micro=5, releaselevel='final', serial=0)
$ uv run --python=3.10 -- test.py
Reading inline script metadata from: test.py
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]]
sys.version_info(major=3, minor=10, micro=14, releaselevel='final', serial=0) |
|
@ptrcnull Is that apk or docker cpython build relocatable, i.e. does it work no matter where we unpack the archive to? One of the main motivations for python-build-standalone was that the upstream cpython builds aren't, see e.g. https://discuss.python.org/t/creating-a-standalone-cpython-distribution/19402. |
|
@konstin Yep you should be fine, see https://github.com/NixOS/patchelf/blob/a0f5433/src/patchelf.cc#L1483 @ptrcnull Cool cool. Also you can do that without PS: In terms of the new apk package format I can guarantee it'll be easily reverse-engineerable! |
|
@konstin looks like it; importing numpy and zlib ( with numpy using its own native libraries and zlib using "system" libz.so ) shows both of them loading libraries from the correct places, with only the system-wide dynamic loader being used: really long terminal outputrpath is doing the heavy lifting here when it comes to "system" libraries like ncurses and zlib, the rest is just Python |
@SamuelMarks yeah, but then you have to resolve the dependency tree yourself and unpack the correct packages, why do that when a Docker container can do that for you? :P still, this is a bodge more than anything, and i don't think this should be used as an actual solution, rather just a proof-of-concept for loading dynamic Python installations under musl
no need for reverse-engineering, it's mostly documented in the repo, and i'm pretty sure a standalone tool for unpacking apk3 packages will appear at some point as well |
As described in #4242, we're currently incorrectly downloading glibc python-build-standalone on musl target, but we also can't fix this by using musl python-build-standalone on musl targets since the musl builds are effectively broken. We reintroduce the libc detection previously removed in #2381, using it to detect which libc is the current one. For simplicity, i've decided to just filter out the musl python-build-standalone archives from the list of available archive, given this is temporary. This means we show the same error message as if we don't have a build for the platform. We could also add a dedicated error message for musl. Fixes #4242 ## Test strategy On my ubuntu host, python downloads continue to pass: ``` target/x86_64-unknown-linux-musl/debug/uv python install ``` On alpine, we fail: ``` $ docker run -it --rm -v .:/io alpine /io/target/x86_64-unknown-linux-musl/debug/uv python install Searching for Python installations error: No download found for request: cpython-any-linux-x86_64-musl ```
As described in #4242, we're currently incorrectly downloading glibc python-build-standalone on musl target, but we also can't fix this by using musl python-build-standalone on musl targets since the musl builds are effectively broken. We reintroduce the libc detection previously removed in #2381, using it to detect which libc is the current one before we have a python interpreter. I changed the strategy a big to support an empty `PATH` which we use in the tests. For simplicity, i've decided to just filter out the musl python-build-standalone archives from the list of available archive, given this is temporary. This means we show the same error message as if we don't have a build for the platform. We could also add a dedicated error message for musl. Fixes #4242 ## Test Plan Tested manually. On my ubuntu host, python downloads continue to pass: ``` target/x86_64-unknown-linux-musl/debug/uv python install ``` On alpine, we fail: ``` $ docker run -it --rm -v .:/io alpine /io/target/x86_64-unknown-linux-musl/debug/uv python install Searching for Python installations error: No download found for request: cpython-any-linux-x86_64-musl ```
As discussed in #4160 (comment)
The text was updated successfully, but these errors were encountered: