pkgs/stdenv/generic/setup.sh taints PATH with unexecutable binaries during cross build #21191
Labels
6.topic: cross-compilation
Building packages on a different platform than they will be used on
Milestone
Comments
joachifm
added
the
6.topic: cross-compilation
|
Should crossDrvs ever even get added to stdenv's In case there is another reason, here are some potential solutions:
|
|
The proper solution is to bite the bullet and only put There currently a bunch of old shitty logic conditionally combining the native and and non-native ones; all that should go. In my ghc-android branch I have some hacks for separate library paths too. |
|
Okay. Do you have any idea why crossDrvs were being put in PATH in the first place? Maybe it was for some binary-compatible derivation dependencies like x86_64 using x86? Or is it just an artifact of the way buildInputs and nativeBuildInputs are being confounded? |
|
The latter. The previous cross stuff was never fully finished + too many cooks in the kitchen. |
Ericson2314
added a commit
to obsidiansystems/nixpkgs
that referenced
this issue
Sep 20, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
bgamari
pushed a commit
to bgamari/nixpkgs
that referenced
this issue
Oct 15, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
ElvishJerricco
pushed a commit
to ElvishJerricco/nixpkgs
that referenced
this issue
Oct 22, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
bgamari
pushed a commit
to bgamari/nixpkgs
that referenced
this issue
Nov 5, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
bgamari
pushed a commit
to bgamari/nixpkgs
that referenced
this issue
Dec 10, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
Ericson2314
added a commit
to obsidiansystems/nixpkgs
that referenced
this issue
Dec 22, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
Ericson2314
added a commit
to obsidiansystems/nixpkgs
that referenced
this issue
Dec 26, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
Ericson2314
added a commit
to obsidiansystems/nixpkgs
that referenced
this issue
Dec 31, 2017
4 far-reaching changes: Smaller PATH, New vars, different propagation logic, and different hook logic Smaller PATH ------------ `buildInputs` no longer go on the PATH at build time, as they cannot be run when cross compiling and we don't want to special case. Simply make a `nativeBuildInput` too if one needs them on the PATH. Fixes NixOS#21191. Many new depedendency variables ------------------------------- See the stdenv chapter of the nixpkgs manual. I pulled out the existing documentation of dependency specification into a new section, and added language for these two (and their propagated equivalents) along side the others'. More complex propagation logic ------------------------------ Before a propagated*XXX*Input always acted as if it was specified directly as a *XXX*Input downstream. That's simple enough, but violates the intended roles of each sort of dep, which has functional and not just stylistic consequences. The new algorithm is detailed in the manual, and ensures everything ends up in the right place. I tried to give both an informal and formal description, but I suspect in practice it will not make much sense until one tries cross compiling, after which it will immediately make sense as the only sane option. Simplified hook logic --------------------- Rather than `envHook` and `crossEnvHook`, whose behavior differs depending on whether we are cross compiling or not, there is now one hook per sort (or rather non-propagated and propagated pair of sorts) of dependency. These new hooks have the same meaning regardless of cross compilation. See the setup hook section of stdenv chapter of the Nixpkgs manual for more details.
|
Tons of spam from the notifications, and this wasn't even automatically closed! :D Might be a good idea to make a test derivation from this, but it definitely works as intended now. |
Ericson2314
added a commit
that referenced
this issue
Jul 24, 2018
Issue #21191 is indeed fixed now.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue description
pkgs/stdenv/generic/setup.sh needs to be smarter about which crossPkgs with a /bin directory it adds to the PATH, specifically
coreutils bash busyboxand probably some others.Steps to reproduce
add
buildInputs = [ coreutils ];and try to build the.crossDrvbasic operations used in a
buildCommandlikecpmkdiretc will fail becausecoreutils.crossDrvhas replaced the native binaries.Technical details
The text was updated successfully, but these errors were encountered: