Compile x86_64 Python with -fPIC

[blowekamp]

I am getting the following linking error when trying to build on with the latest Docker image:

/opt/rh/devtoolset-2/root/usr/libexec/gcc/x86_64-CentOS-linux/4.8.2/ld: /opt/2.7mu/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/opt/2.7mu/lib/libpython2.7.a: could not read symbols: Bad value

[njsmith]

What are you building and how? I believe that in the python configuration we're using in the docker image, regular python extension modules shouldn't be linked against -lpython2.7, and distutils shouldn't do that by default.

[blowekamp]

I am a developer for the SimpleITK project. It uses CMake to build the library and python module. And then distutil/setuptools to package the resulting binary library.

I have update the build files not to link against libpython and the resulting wheel passes auditwheel.

Interestingly, if I use quay.io/pypa/manylinux1_i686 and explicitly link against libpython.a, ( -fPIC is not needed with 32-bit binaries) audit wheel still shows it as compliant. There are other oddities on that platform I still need to investigate.

[njsmith]

It uses CMake to build the library and python module

Ah, I see. This is very confusing (at least it confused us :-), see #30 and this thread), but the short version is that by if your Python was built with --enable-shared (as on, e.g., Fedora and RHEL) then by default distutils will link against libpython<version>.so, and if your Python wasn't built with --enable-shared (as on, e.g., Debian and Ubuntu) then by default it won't. Binaries that arent linked to libpython<version>.so work correctly when loaded into all Pythons; binaries that are linked to libpython<version>.so can break on Debian/Ubuntu. So while there are some obscure reasons why linking to libpython<version>.so should theoretically be better, in practice not linking to it is the portable thing. I guess your CMake code was developed on Fedora/RHEL and no-one ever noticed the bug... if you want to be fancy then you could teach CMake to pull the local configuration out of distutils, but just unconditionally never linking to libpython is also a reasonable choice (certainly better than unconditionally linking to libpython).

Interestingly, if I use quay.io/pypa/manylinux1_i686 and explicitly link against libpython.a, ( -fPIC is not needed with 32-bit binaries) audit wheel still shows it as compliant.

Linking against libpython.a is something else again: this will effectively put a copy of the Python interpreter into your extension module. This will probably work, but is a big waste of space, and is almost certainly not what you want.

Also on i686 then make sure you're running everything under linux32, e.g. docker run ... quay.io/pypa/manylinux1_i686 /bin/linux32 /bin/bash -- this prevents various platform detection code from getting confused about the x86-64 host and the i686 container.

Anyway, sounds like the immediate issue has been solved so I'll mark this closed -- feel free to re-open or open a new issue if you run into anything else.

[blowekamp]

Thank you for your explanation and links.