Build fails for STM32/ARMv7 target

[tronje]

Version(s) of meta-rust

Current master, 81d0fbf

Using Rust 1.78.

Version(s) of poky and/or oe-core

kirkstone-4.0.17, commit 6d1a878bbf24c66f7186b270f823fcdf82e35383

Expected result

The rust-hello-world recipe builds successfully

Actual result

Trying to build rust-hello-world does not actually reach trying to build that recipe. Instead, building libstd-rs fails.
Abbreviated error output:

|      Running `rustc --crate-name libc --edition=2015 /workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/rustc-1.78.0-src/vendor/libc/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="align"' --cfg 'feature="rustc-dep-of-std"' --cfg 'feature="rustc-std-workspace-core"' -C metadata=ffe5355565292cd6 -C extra-filename=-ffe5355565292cd6 --out-dir /workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/build/arm-poky-linux-gnueabi/release/deps --target arm-poky-linux-gnueabi -C linker=/workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/wrapper/target-rust-ccld -L dependency=/workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/build/arm-poky-linux-gnueabi/release/deps -L dependency=/workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/build/release/deps --extern rustc_std_workspace_core=/workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/build/arm-poky-linux-gnueabi/release/deps/librustc_std_workspace_core-7df3a077613c532b.rmeta --cap-lints allow -L /workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/recipe-sysroot/usr/lib/rust --remap-path-prefix=/workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0=/usr/src/debug/libstd-rs/1.78.0-r0 -Cembed-bitcode=yes -L /workdir/build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/recipe-sysroot/usr/lib -C link-arg=-Wl,-soname,libstd.so --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_int128 --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_non_exhaustive --cfg libc_long_array --cfg libc_ptr_addr_of --cfg libc_underscore_const_names --cfg libc_thread_local --cfg libc_const_extern_fn`
| error: data-layout for target `arm-poky-linux-gnueabi`, `e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64`, differs from LLVM target's `armv7-unknown-linux-gnueabihf` default layout, `e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64`

Basically the data-layout is different between Rust's and LLVM's, I guess...?

Steps to reproduce

I'm targeting a machine that uses an STM32MP1. Board support, etc. is from here: https://github.com/karo-electronics/meta-karo
I guess try to build rust-hello-world with current meta-rust for a machine that uses the armv7-eabi data layout...

Additional information

Building the rust-hello-world recipe works fine with my Poky's version of Rust, and that, curiously, has the same DATA_LAYOUT set as meta-rust. I thought maybe the difference in version of rust-llvm might be the issue, but as far as I can tell, both versions of rust-llvm use the same data layout, too. I tried simply pasting the LLVM data layout from the error message into rust-common.inc, but to no avail. I did a bitbake -c cleanall rust-native rust-llvm-native, but that did not help, either..

I did notice b93853c which fixes basically the same issue but for x86.

Any help is appreciated, thanks!

Update

I should have mentioned that all this happened while trying to build with Rust 1.78.0. I've added that information above.

I tried using Rust 1.75.0, and that fails at a different stage:

|      Running `/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/bootstrap/debug/rustc --crate-name rustc_baked_icu_data --edition=2021 compiler/rustc_baked_icu_data/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -Zunstable-options --check-cfg 'values(feature, "rustc_use_parallel_compiler")' --check-cfg 'names()' --check-cfg 'values()' -C metadata=d80321c1703c5ddc -C extra-filename=-d80321c1703c5ddc --out-dir /workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps --target x86_64-linux -C linker=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/wrapper/target-rust-cc -L dependency=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps -L dependency=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps --extern icu_list=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps/libicu_list-cda5c769a969b9c4.rmeta --extern icu_locid=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps/libicu_locid-a0307f8999234a04.rmeta --extern icu_locid_transform=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps/libicu_locid_transform-861103e127456362.rmeta --extern icu_provider=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps/libicu_provider-6a3919b9e9a2d926.rmeta --extern icu_provider_adapters=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps/libicu_provider_adapters-c0d119e935491acb.rmeta --extern zerovec=/workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-linux/release/deps/libzerovec-f28d2624ac7f349e.rmeta --cfg=windows_raw_dylib -Csymbol-mangling-version=v0 -Zunstable-options '--check-cfg=values(bootstrap)' '--check-cfg=values(parallel_compiler)' '--check-cfg=values(no_btreemap_remove_entry)' '--check-cfg=values(crossbeam_loom)' '--check-cfg=values(span_locations)' '--check-cfg=values(rustix_use_libc)' '--check-cfg=values(emulate_second_only_system)' '--check-cfg=values(windows_raw_dylib)' -Zdual-proc-macros -Zmacro-backtrace -Clink-args=-Wl,-z,origin '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Csplit-debuginfo=off -Zunstable-options '-Wrustc::internal' -Cprefer-dynamic -Z binary-dep-depinfo`
| error[E0308]: mismatched types
|     --> /workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/compiler/rustc_index/src/lib.rs:47:32
|      |
| 45   | macro_rules! static_assert_size {
|      | ------------------------------- in this expansion of `static_assert_size!`
| 46   |     ($ty:ty, $size:expr) => {
| 47   |         const _: [(); $size] = [(); ::std::mem::size_of::<$ty>()];
|      |                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an array with a fixed size of 24 elements, found one with 32 elements
|      |
|     ::: compiler/rustc_ast/src/ast.rs:3163:34
|      |
| 3163 |     static_assert_size!(LitKind, 24);
|      |     --------------------------------
|      |     |                            |
|      |     |                            help: consider specifying the actual array length: `32`
|      |     in this macro invocation
|
| error[E0308]: mismatched types
|     --> /workdir/build/tmp/work/x86_64-linux/rust-native/1.75.0-r0/rustc-1.75.0-src/compiler/rustc_index/src/lib.rs:47:32
|      |
| 45   | macro_rules! static_assert_size {
|      | ------------------------------- in this expansion of `static_assert_size!`
| 46   |     ($ty:ty, $size:expr) => {
| 47   |         const _: [(); $size] = [(); ::std::mem::size_of::<$ty>()];
|      |                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an array with a fixed size of 40 elements, found one with 48 elements
|      |
|     ::: compiler/rustc_ast/src/ast.rs:3165:38
|      |
| 3165 |     static_assert_size!(MetaItemLit, 40);
|      |     ------------------------------------
|      |     |                                |
|      |     |                                help: consider specifying the actual array length: `48`
|      |     in this macro invocation
|

The above error occurs while trying to build rust-native.

[compenguy]

I have also started running into the error that @tronje reports in their update, but on an imx6 build of rust-native 1.73. I can provide additional details if helpful - I just wanted to help clarify that this isn't limited to a niche build setting or target.

[karlri]

I can confirm both of the above issues in imx8. 1.78 builds and works if you do a manual edit of the file usr/lib/rustlib/arm-poky-linux-gnueabi.json and rebuild libstd-rs. Changes to DATA_LAYOUT[armv7-eabi] = in rust-common.inc do not seem to take. libstd-rs builds on 1.78 after manually editing usr/lib/rustlib/arm-poky-linux-gnueabi.json in accordance with the data layout mismatch error though.

[tronje]

I can confirm both of the above issues in imx8. 1.78 builds and works if you do a manual edit of the file usr/lib/rustlib/arm-poky-linux-gnueabi.json and rebuild libstd-rs. Changes to DATA_LAYOUT[armv7-eabi] = in rust-common.inc do not seem to take. libstd-rs builds on 1.78 after manually editing usr/lib/rustlib/arm-poky-linux-gnueabi.json in accordance with the data layout mismatch error though.

That's interesting. There are two of these json files in my build directory, do you know which one is the one I need to change? Or simply both of them?

$ fd arm-poky-linux-gnueabi.json build
build/tmp/work/x86_64-linux/rust-cross-cortexa7t2hf-neon-vfpv4-glibc/1.78.0-r0/targets/arm-poky-linux-gnueabi.json
build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/recipe-sysroot-native/usr/lib/rustlib/arm-poky-linux-gnueabi.json

[karlri]

Both probably would not hurt but the latter would be my guess. It's just a problem when targeting arm. The first one would probably be a native build of the rust compiler that should run on x86_64 and produce code trgeting arm. The latter would be a build targeting arm so the file will be used in that case and cause issues for libstd-rs to build.

[karlri]

Now how and why changes to DATA_LAYOUT in rust-common.inc do not take and propagate to the file arm-poky-linux-gnueabi.json I do not understand. It seems to be generated by rust_gen_target in rust-common.inc which takes the string from DATA_LAYOUT.

[karlri]

Modifying the json file like this is only a temporary fix and if you delete the tmp directory and rebuild the error is back. I did lik this:

1. modify DATA_LAYOUT for armv7 in rust-common.inc
2. bitbake libstd-rs, wait for it to fail
3. edit "data-layout" the arm-poky-linux-gnueabi.json file (which I think should have been generated in accordance with DATA_LAYOUT in rust-common.inc but for some reason that is not the case)
4. bitbake libstd-rs again, success

[tronje]

Yeah, I figured any change to some generated file would only be a temporary fix, but at least it would build for the time being. I'll give it a shot later today or tomorrow, probably. Thanks a lot @karlri 🙂

[tronje]

Okay, so after fixing the data-layout in build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/libstd-rs/1.78.0-r0/recipe-sysroot-native/usr/lib/rustlib/arm-poky-linux-gnueabi.json, libstd-rs built but rust-hello-world then failed with the same error. Seems like meta-rust generates the target JSON for every recipe, so I also fixed the data-layout in build/tmp/work/cortexa7t2hf-neon-vfpv4-poky-linux-gnueabi/rust-hello-world/git-r0/recipe-sysroot-native/usr/lib/rustlib/arm-poky-linux-gnueabi.json, and then rust-hello-world built successfully.

So I suppose that means this requires a two part fix.
Part one would be:

diff --git a/recipes-devtools/rust/rust-common.inc b/recipes-devtools/rust/rust-common.inc
index 5196972..47a6d68 100644
--- a/recipes-devtools/rust/rust-common.inc
+++ b/recipes-devtools/rust/rust-common.inc
@@ -130,7 +130,7 @@ MAX_ATOMIC_WIDTH[arm-eabi] = "64"
 FEATURES[arm-eabi] = "+v6,+vfp2"

 ## armv7-unknown-linux-gnueabihf
-DATA_LAYOUT[armv7-eabi] = "e-m:e-p:32:32-i64:64-v128:64:128-a:0:32-n32-S64"
+DATA_LAYOUT[armv7-eabi] = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
 LLVM_TARGET[armv7-eabi] = "${RUST_TARGET_SYS}"
 TARGET_ENDIAN[armv7-eabi] = "little"
 TARGET_POINTER_WIDTH[armv7-eabi] = "32"

And part two is fixing whatever is causing this change to not have any effect...

[tronje]

I added some simple logging to rust_gen_target in rust-common.inc, and that seemed to trigger a rebuild of the Rust toolchain. Now everything built fine (with the fixed data layout). Maybe the reason that changing the data layout in rust-common.inc had no effect was some Yocto magic. I don't quite understand why my -c cleanall invocations did not have that effect, though.

[tronje]

I removed my logging from rust_gen_target, and then my builds started failing again because they were using the incorrect data layout. Now I'm really lost 🙃

edit: after adding logging back to rust_gen_target, bitbake rebuilds the Rust toolchain and the build ultimately succeeds again. I guess there's some cached output of rust_gen_target somewhere, and that's not used when I add logging because the hash of the function would have changed, but it is used again whenever I restore the function to its original state. I thought bitbake hashes whole files, but I guess individual functions, too?

Caching function output this aggressively seems weird, especially since functions have access to an environment full of a myriad of variables. Maybe I'm wrong though and it's something else? No idea what it could be, though.

edit 2: of course, rust_gen_target is called by do_rust_gen_targets, and that is added as a task, and then it makes sense that bitbake keeps cached artifacts for this. Kind of.

[tronje]

I'm guessing this is the fix:

diff --git a/recipes-devtools/rust/rust-common.inc b/recipes-devtools/rust/rust-common.inc
index 47a6d68..5ec9763 100644
--- a/recipes-devtools/rust/rust-common.inc
+++ b/recipes-devtools/rust/rust-common.inc
@@ -378,4 +378,11 @@ python do_rust_gen_targets () {

 addtask rust_gen_targets after do_patch before do_compile
 do_rust_gen_targets[dirs] += "${WORKDIR}/targets"
-
+do_rust_gen_targets[vardeps] += "\
+    LLVM_TARGET \
+    DATA_LAYOUT \
+    MAX_ATOMIC_WIDTH \
+    TARGET_POINTER_WIDTH \
+    TARGET_C_INT_WIDTH \
+    TARGET_ENDIAN \
+"

Testing this now...

edit: I guess either bitbake -c cleanall libstd-rs or bitbake -f -c rust_gen_targets libstd-rs would also have fixed my issue, but I never did either, because I assumed this issue to lie with the Rust toolchain itself.

[karlri]

That totally makes sense, great analysis. Could it be that the same change should be made to arm-eabi aswell?

[tronje]

That totally makes sense, great analysis.

Thanks 🙂

Could it be that the same change should be made to arm-eabi aswell?

rustup run nightly rustc -Z unstable-options --print target-spec-json --target arm-unknown-linux-gnueabihf says:

  "data-layout": "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64",

...so I guess so. Will add it to #446.

[kraj]

This is infact reflecting - llvm/llvm-project@308e82e
The LLVM data-layout is also documented here - https://llvm.org/docs/LangRef.html#data-layout

Change looks ok to me, however, I wonder how this descrepency was detected now and not before. alignment for function pointers was added to spec long time ago ( llvm9 )

FWIW. rust in oe-core has same problem seemingly.