NET8 Android binding library generated sources are deleted when app resources are updated #8658

uholeschak · 2024-01-21T22:36:03Z

Android application type

.NET Android (net7.0-android, net8.0-android, etc.)

Affected platform version

VS2022 17.8.5

Description

The NET8 Android binding library generates it sources files at obj\$(Configuration)\$(TargetFramework)\generated\src.
When the resources of the corresponding app are updated by MSBuild:UpdateGeneratedFiles the msbuild target PrepareResources is called, which deletes the generated sources.
When debugging this causes problems, because the source has changed.
Building the application fails if resources files are open.

Steps to Reproduce

Create NET8 Android binding library for an .AAR file
Use the binding library in a application as project reference
Build the application
Check the binding library generated source at obj\$(Configuration)\$(TargetFramework)\generated\src
Open the app Android resource and change it
Watch obj\$(Configuration)\$(TargetFramework)\generated\src, the source will be deleted after some delay.
While debugging (hitting a break point) the hot reload window is displayed, because the source files have been deleted.

Did you find any workaround?

Create a file Directory.Build.targets in the Android binding library project directory

<Project>
    <Target Name="PrepareResources" DependsOnTargets="$(PrepareResourcesDependsOn)" Condition="'$(DesignTimeBuild)'!='true'">
        <Message Text="PrepareResources modified" Importance="High" />
    </Target>
</Project>

Add to all projects in the solution:

    <DisableFastUpToDateCheck>true</DisableFastUpToDateCheck>

This disables the resource update in the Android binding library

Relevant log output

No response

The text was updated successfully, but these errors were encountered:

dellis1972 · 2024-01-22T09:05:31Z

@uholeschak

Are you able to provide binlogs which demonstrates the issue?
You can follow the instrucitons at https://github.com/dotnet/msbuild/blob/main/documentation/wiki/Providing-Binary-Logs.md#capturing-binary-logs-through-visual-studio. This will capture ALL data including the design time builds, it will allow us to see what is happening. Note it is important that we do get the design time build logs, as from what you describe the problem is with one of those.

uholeschak · 2024-01-22T16:01:00Z

Ich have created some log files with Project System Tools 2022 in design mode when opening an Android resource:
VSLogFiles.zip
The interesting files are the 3 log files from the NetBinding projects.
The generated sources are deleted in this case.

dellis1972 · 2024-01-22T16:48:08Z

@uholeschak

Cam you share the csproj for one of those files? Usually I'd extract it from the binlog but I cannot open it on the mac as the https://live.msbuildlog.com/ site has not been updated it seems :(.

Looks like the problem is the _ClearGeneratedManagedBindings target. Its currently using the following condition
'@(InputJar->Count())' == '0' And '@(EmbeddedJar->Count())' == '0' And '@(LibraryProjectZip->Count())' == '0' And '@(_JavaBindingSource->Count())' == '0'.

All of which are evaluating to 0. Something is missing from the conditon on this target it seems.

uholeschak · 2024-01-22T16:52:01Z

Here is one of the binding project files.
UsbSerialNetBinding.zip
It also contains a pre build step, that is irrelevant in this case. It only tries to force a regeneration of the source if the source is missing ...
The input file is an .aar that is auto detected by the project.

Saratsin · 2024-02-05T15:19:37Z

@uholeschak

Cam you share the csproj for one of those files? Usually I'd extract it from the binlog but I cannot open it on the mac as the live.msbuildlog.com site has not been updated it seems :(.

Looks like the problem is the _ClearGeneratedManagedBindings target. Its currently using the following condition '@(InputJar->Count())' == '0' And '@(EmbeddedJar->Count())' == '0' And '@(LibraryProjectZip->Count())' == '0' And '@(_JavaBindingSource->Count())' == '0'.

All of which are evaluating to 0. Something is missing from the conditon on this target it seems.

Shouldn't we check here for @(AndroidLibrary->Count()) == '0' as well?
I mean it's the new way to define a native library dependency for binding projects since net6.0.
I suppose they just forgot to add it to this target.

P.S. As a workaround, try to use old nodes instead of AndroidLibrary in your csproj file (e.g., LibraryProjectZip or EmbeddedJar). At least it fixed an issue in my project

jonathanpeppers · 2024-02-05T16:15:51Z

Shouldn't we check here for @(AndroidLibrary->Count()) == '0' as well?

No, the new item group is simply translated into the "old" names:

https://github.com/xamarin/xamarin-android/blob/06b1d7f82143fb32b1397b3657de5b22fdcfdf7c/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.AvailableItems.targets#L65-L87

It was done this way for compatibility w/ Xamarin.Android -- and the fact we shared code between .NET 6 and Xamarin.Android.

Saratsin · 2024-02-05T20:56:30Z

Shouldn't we check here for @(AndroidLibrary->Count()) == '0' as well?

No, the new item group is simply translated into the "old" names:

https://github.com/xamarin/xamarin-android/blob/06b1d7f82143fb32b1397b3657de5b22fdcfdf7c/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.AvailableItems.targets#L65-L87

It was done this way for compatibility w/ Xamarin.Android -- and the fact we shared code between .NET 6 and Xamarin.Android.

To be honest it's not quite clear that _CategorizeAndroidLibraries target is called during the same MSBuild process when _ClearGeneratedManagedBindings target is called. I suppose that AndroidLibrary nodes might not be translated into older names due to this reason. I just cannot find the other reason of why changing node to LibraryProjectZip worked out

dellis1972 · 2024-02-06T11:07:46Z

@Saratsin

do you have a diagnostic build log which shows the issue? (via aka.ms/binlog).
It would help to see which ItemGroups your project is picking up.

dellis1972 · 2024-02-07T21:46:38Z

ok good news is I think I repo'd the issue.

The new FastUpdate check is what is causing it. It runs some targets which do not play well with the .NET android build system. Working on a fix now. in the mean time adding
<DisableFastUpToDateCheck>true</DisableFastUpToDateCheck>
to the project will work around the issue.

uholeschak · 2024-02-07T22:18:10Z

If you read my first text you see that i have already set <DisableFastUpToDateCheck>true</DisableFastUpToDateCheck> in the project, which was not enough to fix the problem.

Domik234 · 2024-03-25T09:40:22Z

I'm not sure if it is the same problem as mentioned in description, but even when "obj\Debug\net8.0-android\generated\src" files are not removed, Intellisense and VS cannot see generated content.

Built binaries contains all the code but when an application project uses dependency on this Android binding project (.NET 8 - ProjectReference or DLL) - it cannot see anything but Android resources.

Shouldn't we check here for @(AndroidLibrary->Count()) == '0' as well?

No, the new item group is simply translated into the "old" names:
https://github.com/xamarin/xamarin-android/blob/06b1d7f82143fb32b1397b3657de5b22fdcfdf7c/src/Xamarin.Android.Build.Tasks/MSBuild/Xamarin/Android/Xamarin.Android.AvailableItems.targets#L65-L87

It was done this way for compatibility w/ Xamarin.Android -- and the fact we shared code between .NET 6 and Xamarin.Android.

To be honest it's not quite clear that _CategorizeAndroidLibraries target is called during the same MSBuild process when _ClearGeneratedManagedBindings target is called. I suppose that AndroidLibrary nodes might not be translated into older names due to this reason. I just cannot find the other reason of why changing node to LibraryProjectZip worked out

@Saratsin 's idea of using EmbeddedJar Include=".." fixes this. Thanks.

Fixes dotnet#8658 Fixes dotnet#8698 The PR fixes an issue where the bindings files in intermediate `generated` folder get deleted on subsequent builds. This causes the entire binding process to run again, slowing down builds. The root fo the problem is the `_ClearGeneratedManagedBindings` target. It was designed to clean out the `generated` folder in the case where no binding libraries were present. However it turns out if was running during a design time build! During design time builds the binding library item groups are not evaluated, so the `_ClearGeneratedManagedBindings` would run.. deleting everything. To fix this issue we only run the `_ClearGeneratedManagedBindings` in a standard build. The good thing is this allowed us time to take a look at our incremental build process for bindings. The binding generator does produce a `.projitems` file which lists all the files it generated. We can use that data to incrementally clean up the `generated` folder of old files. This way code that is no longer needed can be removed. While we know the linker will eventually remove unused classes and types, it is better to not have to compile this stuff in the first place.

uholeschak · 2024-03-25T17:14:18Z

Thanks, I have now also added to fix the problem:
<LibraryProjectZip Include="..." />

) Fixes: #8658 Fixes: #8698 Design-time builds don't play nicely with binding project builds: % dotnet new androidlib % cat > Example.java <<EOF package e; public class Example { public static void e() { } } EOF % dotnet build -p:DesignTimeBuild=true -v:diag After this initial Design-Time build, we have the following generated source code for the binding: % find obj -iname \*.cs | xargs ls -l -rw-r--r-- 1 user staff 197 Mar 25 19:22 obj/Debug/net8.0-android/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs -rw-r--r-- 1 user staff 441 Mar 25 19:22 obj/Debug/net8.0-android/__Microsoft.Android.Resource.Designer.cs -rw-r--r-- 1 user staff 2975 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/E.Example.cs -rw-r--r-- 1 user staff 1518 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/Java.Interop.__TypeRegistrations.cs -rw-r--r-- 1 user staff 696 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/__NamespaceMapping__.cs -rw-r--r-- 1 user staff 1094 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.AssemblyInfo.cs -rw-r--r-- 1 user staff 407 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.GlobalUsings.g.cs Run a Design-Time build *again*: % dotnet build -p:DesignTimeBuild=true -v:diag …and we're now missing files (?!): % find obj -iname \*.cs | xargs ls -l -rw-r--r-- 1 user staff 197 Mar 25 19:22 obj/Debug/net8.0-android/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs -rw-r--r-- 1 user staff 441 Mar 25 19:22 obj/Debug/net8.0-android/__Microsoft.Android.Resource.Designer.cs -rw-r--r-- 1 user staff 1094 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.AssemblyInfo.cs -rw-r--r-- 1 user staff 407 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.GlobalUsings.g.cs In particular, `$(IntermediateOutputPath)generated/*/**.cs` is gone, including `E.Example.cs`! The result of this is that Design-Time builds and "normal" builds "fight" each other, constantly generating and deleting files, slowing down incremental builds. The root of the problem is the `_ClearGeneratedManagedBindings` target: It was designed to clean out the `generated` folder in the case where no binding libraries were present. However, it turns out it was running during a design time build! During design time builds the binding library item groups are not evaluated, so the `_ClearGeneratedManagedBindings` target would run, deleting everything. Fix this by ensuring we only run the `_ClearGeneratedManagedBindings` target in in "standard"/*non*-Design-Time builds.

Fixes: dotnet#8658 Fixes: dotnet#8698 Design-time builds don't play nicely with binding project builds: % dotnet new androidlib % cat > Example.java <<EOF package e; public class Example { public static void e() { } } EOF % dotnet build -p:DesignTimeBuild=true -v:diag After this initial Design-Time build, we have the following generated source code for the binding: % find obj -iname \*.cs | xargs ls -l -rw-r--r-- 1 user staff 197 Mar 25 19:22 obj/Debug/net8.0-android/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs -rw-r--r-- 1 user staff 441 Mar 25 19:22 obj/Debug/net8.0-android/__Microsoft.Android.Resource.Designer.cs -rw-r--r-- 1 user staff 2975 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/E.Example.cs -rw-r--r-- 1 user staff 1518 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/Java.Interop.__TypeRegistrations.cs -rw-r--r-- 1 user staff 696 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/__NamespaceMapping__.cs -rw-r--r-- 1 user staff 1094 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.AssemblyInfo.cs -rw-r--r-- 1 user staff 407 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.GlobalUsings.g.cs Run a Design-Time build *again*: % dotnet build -p:DesignTimeBuild=true -v:diag …and we're now missing files (?!): % find obj -iname \*.cs | xargs ls -l -rw-r--r-- 1 user staff 197 Mar 25 19:22 obj/Debug/net8.0-android/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs -rw-r--r-- 1 user staff 441 Mar 25 19:22 obj/Debug/net8.0-android/__Microsoft.Android.Resource.Designer.cs -rw-r--r-- 1 user staff 1094 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.AssemblyInfo.cs -rw-r--r-- 1 user staff 407 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.GlobalUsings.g.cs In particular, `$(IntermediateOutputPath)generated/*/**.cs` is gone, including `E.Example.cs`! The result of this is that Design-Time builds and "normal" builds "fight" each other, constantly generating and deleting files, slowing down incremental builds. The root of the problem is the `_ClearGeneratedManagedBindings` target: It was designed to clean out the `generated` folder in the case where no binding libraries were present. However, it turns out it was running during a design time build! During design time builds the binding library item groups are not evaluated, so the `_ClearGeneratedManagedBindings` target would run, deleting everything. Fix this by ensuring we only run the `_ClearGeneratedManagedBindings` target in in "standard"/*non*-Design-Time builds.

) Fixes: #8658 Fixes: #8698 Design-time builds don't play nicely with binding project builds: % dotnet new androidlib % cat > Example.java <<EOF package e; public class Example { public static void e() { } } EOF % dotnet build -p:DesignTimeBuild=true -v:diag After this initial Design-Time build, we have the following generated source code for the binding: % find obj -iname \*.cs | xargs ls -l -rw-r--r-- 1 user staff 197 Mar 25 19:22 obj/Debug/net8.0-android/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs -rw-r--r-- 1 user staff 441 Mar 25 19:22 obj/Debug/net8.0-android/__Microsoft.Android.Resource.Designer.cs -rw-r--r-- 1 user staff 2975 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/E.Example.cs -rw-r--r-- 1 user staff 1518 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/Java.Interop.__TypeRegistrations.cs -rw-r--r-- 1 user staff 696 Mar 25 19:22 obj/Debug/net8.0-android/generated/src/__NamespaceMapping__.cs -rw-r--r-- 1 user staff 1094 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.AssemblyInfo.cs -rw-r--r-- 1 user staff 407 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.GlobalUsings.g.cs Run a Design-Time build *again*: % dotnet build -p:DesignTimeBuild=true -v:diag …and we're now missing files (?!): % find obj -iname \*.cs | xargs ls -l -rw-r--r-- 1 user staff 197 Mar 25 19:22 obj/Debug/net8.0-android/.NETCoreApp,Version=v8.0.AssemblyAttributes.cs -rw-r--r-- 1 user staff 441 Mar 25 19:22 obj/Debug/net8.0-android/__Microsoft.Android.Resource.Designer.cs -rw-r--r-- 1 user staff 1094 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.AssemblyInfo.cs -rw-r--r-- 1 user staff 407 Mar 25 19:22 obj/Debug/net8.0-android/gxa-8706.GlobalUsings.g.cs In particular, `$(IntermediateOutputPath)generated/*/**.cs` is gone, including `E.Example.cs`! The result of this is that Design-Time builds and "normal" builds "fight" each other, constantly generating and deleting files, slowing down incremental builds. The root of the problem is the `_ClearGeneratedManagedBindings` target: It was designed to clean out the `generated` folder in the case where no binding libraries were present. However, it turns out it was running during a design time build! During design time builds the binding library item groups are not evaluated, so the `_ClearGeneratedManagedBindings` target would run, deleting everything. Fix this by ensuring we only run the `_ClearGeneratedManagedBindings` target in in "standard"/*non*-Design-Time builds. Co-authored-by: Dean Ellis <[email protected]>

uholeschak added Area: App+Library Build Issues when building Library projects or Application projects. needs-triage Issues that need to be assigned. labels Jan 21, 2024

uholeschak assigned dellis1972 and jonathanpeppers Jan 21, 2024

dellis1972 added need-info Issues that need more information from the author. and removed needs-triage Issues that need to be assigned. labels Jan 22, 2024

microsoft-github-policy-service bot added need-attention A xamarin-android contributor needs to review and removed need-info Issues that need more information from the author. labels Jan 22, 2024

dellis1972 added this to the Under Consideration milestone Jan 22, 2024

dellis1972 added need-info Issues that need more information from the author. and removed need-attention A xamarin-android contributor needs to review labels Jan 22, 2024

microsoft-github-policy-service bot added need-attention A xamarin-android contributor needs to review and removed need-info Issues that need more information from the author. labels Jan 22, 2024

dellis1972 mentioned this issue Feb 7, 2024

Add test for Binding Projects Incremental Build #8706

Merged

dellis1972 mentioned this issue Feb 8, 2024

Java source file set build-action "AndoidJavaSource" in project causes compile error #8698

Closed

jpobst removed the need-attention A xamarin-android contributor needs to review label Mar 20, 2024

jonpryor closed this as completed in #8706 Mar 26, 2024

jonathanpeppers mentioned this issue Mar 26, 2024

[release/8.0.2xx] [Xamarin.Android.Build.Tasks] DTBs should not rm generator output #8835

Merged

github-actions bot locked and limited conversation to collaborators Apr 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NET8 Android binding library generated sources are deleted when app resources are updated #8658

NET8 Android binding library generated sources are deleted when app resources are updated #8658

uholeschak commented Jan 21, 2024

dellis1972 commented Jan 22, 2024

uholeschak commented Jan 22, 2024 •

edited

Loading

dellis1972 commented Jan 22, 2024

uholeschak commented Jan 22, 2024 •

edited

Loading

Saratsin commented Feb 5, 2024 •

edited

Loading

jonathanpeppers commented Feb 5, 2024

Saratsin commented Feb 5, 2024

dellis1972 commented Feb 6, 2024

dellis1972 commented Feb 7, 2024

uholeschak commented Feb 7, 2024

Domik234 commented Mar 25, 2024

uholeschak commented Mar 25, 2024

NET8 Android binding library generated sources are deleted when app resources are updated #8658

NET8 Android binding library generated sources are deleted when app resources are updated #8658

Comments

uholeschak commented Jan 21, 2024

Android application type

Affected platform version

Description

Steps to Reproduce

Did you find any workaround?

Relevant log output

dellis1972 commented Jan 22, 2024

uholeschak commented Jan 22, 2024 • edited Loading

dellis1972 commented Jan 22, 2024

uholeschak commented Jan 22, 2024 • edited Loading

Saratsin commented Feb 5, 2024 • edited Loading

jonathanpeppers commented Feb 5, 2024

Saratsin commented Feb 5, 2024

dellis1972 commented Feb 6, 2024

dellis1972 commented Feb 7, 2024

uholeschak commented Feb 7, 2024

Domik234 commented Mar 25, 2024

uholeschak commented Mar 25, 2024

uholeschak commented Jan 22, 2024 •

edited

Loading

uholeschak commented Jan 22, 2024 •

edited

Loading

Saratsin commented Feb 5, 2024 •

edited

Loading