[mono][aot] `.aotdata` files size increase compared to _legacy_ mono #57689

spouliot · 2021-08-18T20:52:47Z

This has been discussed a few times, in chats and meetings, but not filed (as far as I know).

It's been noted since the early net6 previews [1] that the .aotdata files are much larger in net6. Even if the overall app size is larger the ratio for .aotdata is also higher.

Example

full data: https://gist.github.com/spouliot/5de7b503f4b398d388a793e0d608ccce

Xamarin.iOS.dll

Directories / Files	Legacy	dotnet	diff	%
Xamarin.iOS.aotdata.arm64	36,448	48,232	11,784	32.3%
Xamarin.iOS.dll	56,832	52,736	-4,096	-7.2%

Here the linked [2] Xamarin.dll is smaller for dotnet than legacy. Their content is not 100% identical but quite close (I doubt it could explain the difference). Even if the assembly is smaller (7.2% smaller) the .aotdata size for dotnet is quite larger (32.3%).

mscorlib.dll and System.Private.CoreLib.dll

This same pattern seems to apply to other assemblies - but their internal differences makes it harder to compare.

Directories / Files	Legacy	dotnet	diff	%
mscorlib.aotdata.arm64	284,864	0	-284,864	-100.0%
mscorlib.dll	407,552	0	-407,552	-100.0%
System.Private.CoreLib.aotdata.arm64	0	731,824	731,824	-
System.Private.CoreLib.dll	0	464,384	464,384	-

The assembly [2] size for corelib for dotnet (464,384) and legacy (407,552) are comparable, but the .aotdata more than twice the size for dotnet.

Questions

Do we know why are the .aotdata files have become larger ?
- if more data was removed from the native executable then the impact has not been seen (since it's also larger than before)
Is there any way to can be reduced ? (likely easier to answer once we know why :-)

[1] dotnet/macios#10249
[2] the assemblies size are all stripped of IL (both legacy and dotnet)

The text was updated successfully, but these errors were encountered:

SamMonoRT · 2021-08-22T22:08:04Z

Looking at numbers being collected against the HelloWorld iOS Sample against main (https://github.com/dotnet/runtime/tree/main/src/mono/sample/iOS), and being reported here https://msit.powerbi.com/groups/me/apps/54e0e83f-07bc-45bf-87b7-a7677ff3af2a/reports/161e76f5-dad3-4b78-b4d2-56db2da6bdc4/ReportSection85643634814afcbdba99 there seems to be some difference in numbers between the ones collected above and ones from the automation in the lab.
1.When opening the above page make sure to set the page view to "Fit to Width" to see it clearly. You can click on each file name to see the size.
2. One common reporting in icu.dat file 2,176,304 (which is a good sign)
3. One biggest difference in size of System.Private.Corelib. dll ( 464,384 (above reporting) v/s 3,582,464) @lambdageek @spouliot @billwert - we need to figure why such a large difference in those)
4. Aotdata files seem smaller in the automation, example System.Private.CoreLib.aotdata (731,824 (above reporting) v/s 2,336,840)

billwert · 2021-08-22T23:25:11Z

Different AOT flags perhaps?

SamMonoRT · 2021-08-23T16:25:37Z

@rolfbjarne fyi for above difference in aotdata size-- we should try to narrow down any differences in flags used

spouliot · 2021-08-23T21:56:25Z

I was running the (work in progress) numbers for RC1 and got

Directories / Files	Legacy	RC1	diff	%
Xamarin.iOS.aotdata.arm64	36,456	33,440	-3,016	-8.3%
Xamarin.iOS.dll	56,832	73,728	16,896	29.7%

Xamarin.iOS.aotdata.arm64 is smaller in release/6.0.1xx-rc.1 than it is on (more recent) main - and the ratio would make (more) sense after stripping the IL [1]

[1] RC1 assembly size is not stripped of IL, while Legacy is.

rolfbjarne · 2021-08-24T08:25:37Z

@rolfbjarne fyi for above difference in aotdata size-- we should try to narrow down any differences in flags used

I checked, and the only difference is that legacy passes -O=-float32 to the AOT Compiler:

Legacy:

/Users/rolf/work/maccore/xharness/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/bin/arm64-darwin-mono-sgen \
	--debug \
	--llvm \
	-O=gsharedvt \
	-O=-float32 \
	--aot=
		mtriple=arm64-ios,data-outfile=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/arm64/Xamarin.iOS.aotdata.arm64,
		static,
		asmonly,
		direct-icalls,
		full,
		nodebug,
		dwarfdebug,
		msym-dir=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/3-Build/Msym,
		llvm-path=/Users/rolf/work/maccore/xharness/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/LLVM/bin/,
		outfile=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/arm64/Xamarin.iOS.dll.s,
		llvm-outfile=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/arm64/Xamarin.iOS.dll-llvm.o \
	/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/3-Build/Xamarin.iOS.dll


.NET:

/Users/rolf/work/maccore/main/xamarin-macios/builds/downloads/dotnet-sdk-6.0.100-rc.1.21417.3-osx-x64/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/6.0.0-rc.1.21415.6/Sdk/../tools/mono-aot-cross \
	 --debug \
	 --llvm \
	 --aot=
	 	mtriple=arm64-ios,
	 	data-outfile=obj/iPhone/Release/net6.0-ios/ios-arm64/nativelibraries/aot-output/arm64/Xamarin.iOS.aotdata,
	 	static,
	 	asmonly,
	 	direct-icalls,
	 	full,
	 	nodebug,
	 	dwarfdebug,
	 	llvm-path=/Users/rolf/work/maccore/main/xamarin-macios/builds/downloads/dotnet-sdk-6.0.100-rc.1.21417.3-osx-x64/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/6.0.0-rc.1.21415.6/Sdk/../tools,
	 	outfile=obj/iPhone/Release/net6.0-ios/ios-arm64/nativelibraries/aot-output/arm64/Xamarin.iOS.dll.s,
	 	llvm-outfile=obj/iPhone/Release/net6.0-ios/ios-arm64/nativelibraries/aot-output/arm64/Xamarin.iOS.dll.llvm.o \
	 -O=gsharedvt \
	 /Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/dotnet/obj/iPhone/Release/net6.0-ios/ios-arm64/linked/Xamarin.iOS.dll

so I tried adding -O=-float32 to the .NET version, but there was no difference in size whatsoever.

SamMonoRT · 2021-08-25T11:25:24Z

@rolfbjarne fyi for above difference in aotdata size-- we should try to narrow down any differences in flags used

I checked, and the only difference is that legacy passes -O=-float32 to the AOT Compiler:

Legacy:

/Users/rolf/work/maccore/xharness/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/bin/arm64-darwin-mono-sgen \
	--debug \
	--llvm \
	-O=gsharedvt \
	-O=-float32 \
	--aot=
		mtriple=arm64-ios,data-outfile=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/arm64/Xamarin.iOS.aotdata.arm64,
		static,
		asmonly,
		direct-icalls,
		full,
		nodebug,
		dwarfdebug,
		msym-dir=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/3-Build/Msym,
		llvm-path=/Users/rolf/work/maccore/xharness/xamarin-macios/_ios-build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/LLVM/bin/,
		outfile=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/arm64/Xamarin.iOS.dll.s,
		llvm-outfile=/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/arm64/Xamarin.iOS.dll-llvm.o \
	/Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/oldnet/obj/iPhone/Release/mtouch-cache/3-Build/Xamarin.iOS.dll


.NET:

/Users/rolf/work/maccore/main/xamarin-macios/builds/downloads/dotnet-sdk-6.0.100-rc.1.21417.3-osx-x64/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/6.0.0-rc.1.21415.6/Sdk/../tools/mono-aot-cross \
	 --debug \
	 --llvm \
	 --aot=
	 	mtriple=arm64-ios,
	 	data-outfile=obj/iPhone/Release/net6.0-ios/ios-arm64/nativelibraries/aot-output/arm64/Xamarin.iOS.aotdata,
	 	static,
	 	asmonly,
	 	direct-icalls,
	 	full,
	 	nodebug,
	 	dwarfdebug,
	 	llvm-path=/Users/rolf/work/maccore/main/xamarin-macios/builds/downloads/dotnet-sdk-6.0.100-rc.1.21417.3-osx-x64/packs/Microsoft.NETCore.App.Runtime.AOT.osx-x64.Cross.ios-arm64/6.0.0-rc.1.21415.6/Sdk/../tools,
	 	outfile=obj/iPhone/Release/net6.0-ios/ios-arm64/nativelibraries/aot-output/arm64/Xamarin.iOS.dll.s,
	 	llvm-outfile=obj/iPhone/Release/net6.0-ios/ios-arm64/nativelibraries/aot-output/arm64/Xamarin.iOS.dll.llvm.o \
	 -O=gsharedvt \
	 /Users/rolf/work/maccore/main/xamarin-macios/tests/dotnet/size-comparison/MySingleView/dotnet/obj/iPhone/Release/net6.0-ios/ios-arm64/linked/Xamarin.iOS.dll

so I tried adding -O=-float32 to the .NET version, but there was no difference in size whatsoever.

@rolfbjarne - @steveisok figured out that the Sample HelloWorld iOS app is using an additional flag "mattr=+crc" which seems to be different than your runs. Is it possible to try adding that to see what kind of data you get. The size ratio difference between the runs is a concern and maybe key to identifying what is causing the large aotdata files.

spouliot · 2021-08-30T15:23:42Z

@SamMonoRT I might be mis-reading https://msit.powerbi.com/groups/me/apps/54e0e83f-07bc-45bf-87b7-a7677ff3af2a/reports/161e76f5-dad3-4b78-b4d2-56db2da6bdc4/ReportSection85643634814afcbdba99 but it seems that it does not run ILLink on the app so the numbers (of files and their sizes) is not helpful to compare anything with what we're producing

One biggest difference in size of System.Private.Corelib. dll ( 464,384 (above reporting) v/s 3,582,464) @lambdageek @spouliot @billwert - we need to figure why such a large difference in those)

Not running the linker means nothing is removed from System.Private.Corelib.dll so it's .aotdata file will be much bugger (same as the .dll itself).

There's value in monitoring the original (input) size of the files across the runtime builds. However those numbers have little relation to what will end up inside the customer applications. At the very least they should not be compared to each other.

SamMonoRT · 2021-08-30T18:36:11Z

@billwert - Is it possible to run the perf automation with linker enabled by uncommenting out lines in

runtime/src/mono/sample/iOS/Program.csproj

Lines 13 to 17 in d033e1a

    
           <PropertyGroup> 
        
             <!-- FIXME: for some reason trimming is throwing errors about something in System.Diagnostics. --> 
        
             <!-- <PublishTrimmed>true</PublishTrimmed> --> 
        
             <!-- <TrimMode>Link</TrimMode> --> 
        
           </PropertyGroup>

?

billwert · 2021-08-30T18:38:19Z

@SamMonoRT Sure I'll do a quick test run and see if it works.

SamMonoRT · 2021-08-30T18:39:22Z

@SamMonoRT I might be mis-reading https://msit.powerbi.com/groups/me/apps/54e0e83f-07bc-45bf-87b7-a7677ff3af2a/reports/161e76f5-dad3-4b78-b4d2-56db2da6bdc4/ReportSection85643634814afcbdba99 but it seems that it does not run ILLink on the app so the numbers (of files and their sizes) is not helpful to compare anything with what we're producing

One biggest difference in size of System.Private.Corelib. dll ( 464,384 (above reporting) v/s 3,582,464) @lambdageek @spouliot @billwert - we need to figure why such a large difference in those)

Not running the linker means nothing is removed from System.Private.Corelib.dll so it's .aotdata file will be much bugger (same as the .dll itself).

There's value in monitoring the original (input) size of the files across the runtime builds. However those numbers have little relation to what will end up inside the customer applications. At the very least they should not be compared to each other.

That makes sense. @lambdageek is taking a look at this issue and mentioned the same to me.

billwert · 2021-08-31T18:26:51Z

Just uncommenting those lines seems to have broken running LLVM, investigating..

imhameed · 2021-09-08T09:39:17Z

A dummy assembly with 10,000 functions of the form public static void DummyFunc9971() { } generates 135kb of aotdata with dotnet/runtime at revision 721a802. mono/mono at revision 6c61baf83c215daf21d67dd2a31244dcf7404bb1 generates 86kb for the same assembly. So there's some extra per-function aotdata size increase compared to legacy mono, although I don't know what to attribute it to yet--all the immediate callers of emit_aot_data look identical in mono/mono and dotnet/runtime.

imhameed · 2021-09-09T01:09:51Z

The effect I observed in my previous comment is not actually relevant. mono_serialize_debug_info will now produce a 5-byte-long "debug info" for LLVM functions--instead of an empty 0-byte-long one as in legacy Mono, and that is why the generated aotdata for my "10,000 empty functions" test increased in size. This doesn't affect xamarin-macios because Release LLVM-enabled builds use --aot=...,nodebug,... which turns this specific subfeature (accumulation of debug info in the aotdata "blob" section) off entirely.

I'm not actually sure that aotdata sizes have increased since legacy Mono. Here's the output of make -j8 compare-size in xamarin-macios/tests/dotnet from revision 1542039f22fb4b2b4d4c6037c7021c6e0d6668bc: https://gist.github.com/imhameed/a76cf90c87ebb9d40f25c245cc43f057
I added /p:MtouchUseLlvm=true to the build-dotnet rule in tests/dotnet/Makefile to have a fair comparison between legacy and netcore. Note that the aotdata sizes for equivalent assemblies (Xamarin.iOS.aotdata.arm64 and MySingleView.aotdata.arm64) have actually decreased since legacy mono.

spouliot · 2021-09-09T13:22:41Z

. mono_serialize_debug_info will now produce a 5-byte-long "debug info" for LLVM functions--instead of an empty 0-byte-long one as in legacy Mono, and that is why the generated aotdata for my "10,000 empty functions" test increased in size.

That makes sense and I think you just nailed it :-)

LLVM was not usable (and blocked from being used) in legacy for debug builds. IOW the legacy sample would not have those extraneous debug data.

I'm not actually sure that aotdata sizes have increased since legacy Mono.

It probably did not. Earlier previews did not use LLVM (not available) so the numbers were even more skewed (when the ratio was found to be widely off expectations).

P7 and RC1 numbers both shows a decrease for Xamarin.iOS.aotdata.arm64. Those are pretty much "on ratio" once IL is stripped from dotnet app bundles (still blocked, but underway).

Note that this was not visible on main since LLVM was not automatically enabled (on release builds) due to bugs (now solved and PR in progress, if not already merge, to make this the default for release builds). Release branches (both P7 and rc1) had a change to the project (similar to the one you made) so we could compare LLVM results.

So, unless there are some things you wish to diagnose further, I think that issue can be closed. Huge thanks for your help and analysis!

dotnet-issue-labeler bot added area-Meta untriaged New issue has not been triaged by the area owner labels Aug 18, 2021

SamMonoRT added area-Codegen-AOT-mono and removed untriaged New issue has not been triaged by the area owner labels Aug 18, 2021

SamMonoRT added this to the 7.0.0 milestone Aug 18, 2021

SamMonoRT removed the area-Meta label Aug 18, 2021

SamMonoRT assigned imhameed Aug 18, 2021

dalexsoto mentioned this issue Aug 19, 2021

[.NET 6] External dotnet issues dotnet/macios#8901

Closed

61 tasks

lambdageek self-assigned this Aug 20, 2021

rolfbjarne mentioned this issue Aug 24, 2021

.NET: Setting MtouchFloat32 (or not) in the project file does nothing dotnet/macios#12524

Closed

imhameed closed this as completed Sep 9, 2021

imhameed mentioned this issue Sep 13, 2021

Xamarin.iOS net6 builds are seeing ~10% larger AOT data sizes #53740

Closed

ghost locked as resolved and limited conversation to collaborators Oct 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[mono][aot] `.aotdata` files size increase compared to _legacy_ mono #57689

[mono][aot] `.aotdata` files size increase compared to _legacy_ mono #57689

spouliot commented Aug 18, 2021

SamMonoRT commented Aug 22, 2021

billwert commented Aug 22, 2021

SamMonoRT commented Aug 23, 2021

spouliot commented Aug 23, 2021

rolfbjarne commented Aug 24, 2021

SamMonoRT commented Aug 25, 2021

spouliot commented Aug 30, 2021

SamMonoRT commented Aug 30, 2021 •

edited

Loading

billwert commented Aug 30, 2021

SamMonoRT commented Aug 30, 2021

billwert commented Aug 31, 2021

imhameed commented Sep 8, 2021

imhameed commented Sep 9, 2021 •

edited

Loading

spouliot commented Sep 9, 2021

[mono][aot] .aotdata files size increase compared to _legacy_ mono #57689

[mono][aot] .aotdata files size increase compared to _legacy_ mono #57689

Comments

spouliot commented Aug 18, 2021

Example

Xamarin.iOS.dll

mscorlib.dll and System.Private.CoreLib.dll

Questions

SamMonoRT commented Aug 22, 2021

billwert commented Aug 22, 2021

SamMonoRT commented Aug 23, 2021

spouliot commented Aug 23, 2021

rolfbjarne commented Aug 24, 2021

SamMonoRT commented Aug 25, 2021

spouliot commented Aug 30, 2021

SamMonoRT commented Aug 30, 2021 • edited Loading

billwert commented Aug 30, 2021

SamMonoRT commented Aug 30, 2021

billwert commented Aug 31, 2021

imhameed commented Sep 8, 2021

imhameed commented Sep 9, 2021 • edited Loading

spouliot commented Sep 9, 2021

[mono][aot] `.aotdata` files size increase compared to _legacy_ mono #57689

[mono][aot] `.aotdata` files size increase compared to _legacy_ mono #57689

SamMonoRT commented Aug 30, 2021 •

edited

Loading

imhameed commented Sep 9, 2021 •

edited

Loading