{i386,x86_64}-apple-ios targets pass too many thousands of arguments to the linker

[gnzlbg]

To reproduce:

git clone https://github.com/rust-lang-nursery/packed_simd
cd packed_simd
# Get Rust target:
rustup target add x86_64-apple-ios
# Build cargo runner:
export RUSTFLAGS='-C link-args=-mios-simulator-version-min=7.0'
rustc ./ci/deploy_and_run_on_ios_simulator.rs -o ios_cargo_runner --verbose
export CARGO_TARGET_X86_64_APPLE_IOS_RUNNER=$(pwd)/ios_cargo_runner
# Build library:
cargo build --target=x86_64-apple-ios
# Build and run tests:
cargo test --target=x86_64-apple-ios

Produces this output (too long to put it here, gist: https://gist.github.com/gnzlbg/4ff458a32ebd56e4d8930aaf766178b4). Linking fails. This is rust-lang-nursery/packed_simd issue: rust-lang/packed_simd#26

cc @michaelwoerister @alexcrichton this might be related to incremental compilation, I see a lot of rcgus.

[alexcrichton]

Hm yes so incremental means that the maximal number of cgus are used, where if you have a crate with N modules it basically translates to N (ish) cgus (afaik at least). As to why that ends up failing the linker... I'm not sure! I could be that an OS limit for the command line is silently blown and while rustc may handle it something else like clang isn't handling it?

[gnzlbg]

where if you have a crate with N modules it basically translates to N (ish) cgus (afaik at least)

The crate has ~2500 modules when compiled with cfg(test)... resulting in ~2500 cgus that must be linked together...

[kennytm]

getconf ARG_MAX on macOS is 256 KiB, whereas this linker command is 419 KiB in total.

We should switch the argument file (cc @args.txt) if the total size of arguments is too long.

(This means we apply Command::very_likely_to_exceed_some_spawn_limit to macOS as well.)

[alexcrichton]

Actually I think this is one of the reasons that this takes so long to compile, we're actually already falling back to an @-file! Enabling some debug logs I get:

 INFO 2018-07-26T01:24:52Z: rustc_codegen_llvm::back::link: command line to linker was too big: Argument list too long (os error 7)
 INFO 2018-07-26T01:24:52Z: rustc_codegen_llvm::back::link: falling back to passing arguments to linker via an @-file
  INFO 2018-07-26T01:24:52Z: rustc_codegen_llvm::back::link: invoking linker "cc" "@/var/folders/_k/gjskx0jj207ccwd0tdxq5nb00000gn/T/rustc3HTPIr/linker-arguments"

(full logs)

I'm not really sure what's going on here :(

Maybe we're blowing some limit in the OSX linker? We probably shouldn't be passing thousands of objects anyway

[alecmocatta]

Not all lds support @args syntax – the macOS ld being one of them I believe. Hence gcc or clang is having to pass the arguments to ld at the command line exhaustively, and is thus itself likely bumping into the ARG_MAX limit.

[kennytm]

Interesting. macOS's ld does allow using a file as input though:

     -filelist file[,dirname]
                 Specifies that the linker should link the files listed in
                 file.  This is an alternative to listing the files on the com-
                 mand line.  The file names are listed one per line separated
                 only by newlines. (Spaces and tabs are assumed to be part of
                 the file name.)  If the optional directory name, dirname is
                 specified, it is prepended to each name in the list file.

[gnzlbg]

I wonder, why I am hitting this when using --target=i386-apple-ios but not when using --target=x86_64-apple-darwin ? Aren't they using the same linker ?

[alecmocatta]

@kennytm Good spot. This is a guess from a quick look, but it looks like clang puts the first n contiguous filenames into the filelist (response file in its jargon); the remainder are passed as args as normal. My guess is that the filename in -Wl,-syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.4.sdk is being put into the filelist; the -L /Users/acrichton/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/x86_64-apple-ios/lib is stopping additions to the filelist; and all the remaining thousands of files are passed as args as normal?

As clang and gcc both pass thru the -filelist, perhaps we could create it before giving it to them? Indeed, gcc doesn't seem to ever create a filelist itself so to use that as the driver of apple's ld would presumably not work if the command was too long.

[michaelwoerister]

The crate has ~2500 modules when compiled with cfg(test)

Depending on whether those modules contain generic code too, it might even be ~5000 cgus.

In theory it should be possible to limit the number of cgus created by incremental compilation, at the expensive of less accurate cache utilization.

[steveklabnik]

Triage: lots of changes have happened, not sure if this reproduces or not

[bjorn3]

Removing

rust/compiler/rustc_codegen_ssa/src/back/command.rs

Lines 133 to 137 in 33a2c24

	// We mostly only care about Windows in this method, on Unix the limits
	// can be gargantuan anyway so we're pretty unlikely to hit them
	if cfg!(unix) {
	return false;
	}

should probably fix this.

[keith]

Also note macOS bumped the limit a lot a few years ago, but recent versions of ld64 also support passing a params file so if we could rely on users having the newer versions that would be fine too