Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump to mono:2018-06 #1836

Merged
merged 110 commits into from
Oct 9, 2018
Merged

Bump to mono:2018-06 #1836

merged 110 commits into from
Oct 9, 2018

Conversation

directhex
Copy link
Contributor

No description provided.

luhenry and others added 30 commits December 11, 2017 10:45
@marek-safar
Copy link
Contributor

@jonpryor could you please review the status

Commit list for mono/mono:

* mono/mono@6e48ad4f7b1 Merge pull request #10905 from monojenkins/backport-pr-10899-to-2018-06
* mono/mono@d050e323ed9 [runtime] Fix undefined pthread_main_np (#10930)
* mono/mono@b549fa7f32a [Reflection] Fix issue with finding types in module using an asterisk as filter criteria
* mono/mono@6c46acfe697 [runtime] Disable stack guard for main thread on osx

Diff: mono/mono@0dffbef...6e48ad4
@jonpryor
Copy link
Member

jonpryor commented Oct 4, 2018

There is API Breakage being reported. This breakage should be addressed.

@marek-safar
Copy link
Contributor

API breakage is not significant.

System.Numerics.Vectors type were replaced with type-forwarders (the tool cannot detect that)

MemoryHandle change is due to previous Span preview which we missed to hide (it should be in previous release only)

CreateAsyncMethodBuilder - is infrastructure code not intended for public usage

@jonpryor
Copy link
Member

jonpryor commented Oct 5, 2018

API breakage is not significant.

I want to believe that.

But then I remember:

Especially after the above string.Concat() experience, do we really think that the MemoryHandle(IRetainable, void*, GCHandle) constructor removal, IRetainable type removal, and CreateAsyncMethodBuilder() removal are really "not significant"?

This certainly suggests that the CreateAsyncMethodBuilder() removal is fine:

Checked API usage (nuget and API port): no usage at all.

I can't easily find anything similar regarding IRetainable or HasPointer, just lots of forks of the corefx repo which contain the IRetainable type, as well as tvOS documentation:

https://github.com/xamarin/documentation/blob/28fe21da3488aced982082fbf72a40393e027cfd/releases/ios/api_changes/tvos-11-10-1-11-11-0.md

IRetainable is even mentioned in the .NET Core 2.0 .. 2.1 diff: https://github.com/dotnet/apireviews/blob/ac765393a7281184ec30948b7a6d86b7ad58af1c/2018/NetCore20-vs-NetCore21-Diff/README.md

Are the changes really not significant?

@marek-safar
Copy link
Contributor

Here is the story behind IRetainable dotnet/corefx@f592e88#diff-577bd970484f00965a5e67de411cab76. It was design change before RTW unfortunately at that point we were not using stable branch of CoreFX so we accidentally shipped that API because we missed it when hiding the unstable Span APIs. I'd be very surprised if anyone managed to take a dependency on it in 15.8 which was the first release the API showed up and 15.9 won't have it either.

@jonpryor
Copy link
Member

jonpryor commented Oct 5, 2018

@marek-safar, @luhenry: Then the next step is to submit a PR against https://github.com/xamarin/xamarin-android-api-compatibility which updates reference/mscorlib.xml and reference/System.Numerics.Vectors.xml with the expected API changes. This can be done from a xamarin-android checkout with:

$ cd external/xamarin-android-api-compatibility
$ make update XA_FRAMEWORK_DIR=$(cd ../../ && pwd)/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid

Once the xamarin-android-api-compatibility PR has been merged, this PR can be updated to include a xamarin-android-api-compatibility bump, which will allow things to work.

@marek-safar
Copy link
Contributor

@luhenry could you run the make update Jonathan asked for ?

jonpryor pushed a commit to xamarin/xamarin-android-api-compatibility that referenced this pull request Oct 8, 2018
Context: dotnet/android#1836
Context: https://jenkins.mono-project.com/job/xamarin-android-pr-builder/4175/API_20Compatibility_20Checks/

The mono/mono:2018-06@70fe915d bump introduced "breaking" changes to
`System.Numerics.Vectors.dll` and `mscorlib.dll`.

`System.Numerics.Vectors.dll` "lost" the `System.Numerics.Vector` and
`System.Numerics.Vector<T>` types, because those were moved to
`mscorlib.dll`, with type forwarders in `System.Numerics.Vectors.dll`.
Unfortunately, `mono-api-html` doesn't understand type forwarders,
and thus reported their movement as breakage.

Update `reference/System.Numerics.Vectors.xml` accordingly.

The `mscorlib.dll` changes are a bit more complicated: the
`System.Buffers.IRetainable` type was removed, as were members
associated with that interface:

	<h3>Removed Type <span class='breaking' data-is-breaking>System.Buffers.IRetainable</span></h3>

	<!-- start type MemoryHandle --> <div>
	<h3>Type Changed: System.Buffers.MemoryHandle</h3>
	<p>Removed constructor:</p>
	<pre>
	  <span class='removed removed-constructor breaking' data-is-breaking>public MemoryHandle (IRetainable, void*, System.Runtime.InteropServices.GCHandle);</span>
	</pre>
	<p>Removed property:</p>
	<pre>
	  <span class='removed removed-property breaking' data-is-breaking>public bool HasPointer { get; }</span>
	</pre>

The `IRetainable` type was [accidentally shipped][0], and shouldn't
have been provided in the first place.

> [IRetainable] was design change before RTW unfortunately at that
> point we were not using stable branch of CoreFX so we accidentally
> shipped that API because we missed it when hiding the unstable
> Span APIs

Additionally, `System.Threading.Tasks.ValueTask<T>.CreateAsyncMethodBuilder()`
was removed:

	<h3>Type Changed: System.Threading.Tasks.ValueTask`1</h3>
	<p>Removed method:</p>
	<pre>
	  <span class='removed removed-method breaking' data-is-breaking>public static System.Runtime.CompilerServices.AsyncValueTaskMethodBuilder&lt;TResult&gt; CreateAsyncMethodBuilder ();</span>
	</pre>

`ValueTask<T>.CreateAsyncMethodBuilder()` was removed because
[no code was found to be using it][1], and thus is deemed to be an 
acceptable ABI break.

Update `reference/mscorlib.xml` accordingly, and also include all the
new APIs that have been added since c550d1b.

[0]: dotnet/android#1836 (comment)
[1]: https://github.com/dotnet/corefx/issues/22171#issuecomment-327254875
@jonpryor
Copy link
Member

jonpryor commented Oct 8, 2018

xamarin/xamarin-android-api-compatibility#19 has been merged.

The external/xamarin-android-api-compatibility submodule reference can now be bumped and included into this PR.

@jonpryor
Copy link
Member

jonpryor commented Oct 8, 2018

...and with luck the Android emulator won't behave badly on the next build...

Commit list for mono/mono:

* mono/mono@ab3c897d685 Merge pull request #10997 from lewurm/2018-06-interp-fixes-for-native-type
* mono/mono@914a62ab2e7 [interp] introduce float R4 stack type
* mono/mono@62f23a83652 [interp] support ntype.ToString ()
* mono/mono@2c48c62e66f [interp] support nfloat.*Infinity
* mono/mono@5c5a48b1449 [interp] support ntype.Equals ()
* mono/mono@13e412ee52a [interp] support ntype.CompareTo ()
* mono/mono@a576a799b16 [interp] fix op_implicit/op_explicit conversions for native types
* mono/mono@1d9378713e6 [sdks/ios] Build libMonoPosixHelper for device architectures, so that zlib-helper.o is created. (#10921) (#10970)

Diff: mono/mono@6e48ad4...ab3c897
Commit list for xamarin/xamarin-android-api-compatibility:

* xamarin/xamarin-android-api-compatibility@7ccb480 Update BCL assemblies for mono/mono:2018-06@70fe915d breakage (#19)

Diff: xamarin/xamarin-android-api-compatibility@c550d1b...7ccb480
@jonpryor jonpryor merged commit 606675b into master Oct 9, 2018
jonpryor added a commit to xamarin/xamarin-android-api-compatibility that referenced this pull request Oct 9, 2018
Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-d15-9/30/API_20Compatibility_20Checks/

The [xamarin-android commit to bump to mono/mono:2018-04@969357ac][0]
introduced an [ABI break in `mscorlib.dll`][1]:

	<h2>Namespace System.Buffers</h2>
	<h3>Removed Type <span class='breaking' data-is-breaking>System.Buffers.IRetainable</span></h3>
	<h3>Removed Type <span class='breaking' data-is-breaking>System.Buffers.MemoryHandle</span></h3>

The `IRetainable` and `MemoryHandle` types were
[accidentally shipped][2], and shouldn't have been provided in the
first place.  Their presence means that the
[System.Memory NuGet Package][3] cannot be used with Xamarin.Android,
as there would be an ambiguous type reference for the type
`System.Buffers.MemoryHandle`, as it's present in *both*
`mscorlib.dll` and the `System.Memory` NuGet package.

For now, the easiest solution is to remove these types from
`mscorlib.dll`, which in turn requires that we update our expected API
accordingly, removing these two types.

Additionally, flush API additions present in the other assemblies that
are tracked in this repo.

[0]: dotnet/android@a9f6574
[1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-d15-9/30/API_20Compatibility_20Checks/
[2]: dotnet/android#1836 (comment)
[3]: https://www.nuget.org/packages/System.Memory/
jonpryor added a commit to xamarin/xamarin-android-api-compatibility that referenced this pull request Oct 9, 2018
Context: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-d15-9/30/API_20Compatibility_20Checks/

The [xamarin-android commit to bump to mono/mono:2018-04@969357ac][0]
introduced an [ABI break in `mscorlib.dll`][1]:

	<h2>Namespace System.Buffers</h2>
	<h3>Removed Type <span class='breaking' data-is-breaking>System.Buffers.IRetainable</span></h3>
	<h3>Removed Type <span class='breaking' data-is-breaking>System.Buffers.MemoryHandle</span></h3>

The `IRetainable` and `MemoryHandle` types were
[accidentally shipped][2], and shouldn't have been provided in the
first place.  Their presence means that the
[System.Memory NuGet Package][3] cannot be used with Xamarin.Android,
as there would be an ambiguous type reference for the type
`System.Buffers.MemoryHandle`, as it's present in *both*
`mscorlib.dll` and the `System.Memory` NuGet package.

For now, the easiest solution is to remove these types from
`mscorlib.dll`, which in turn requires that we update our expected API
accordingly, removing these two types.

Additionally, flush API additions present in the other assemblies that
are tracked in this repo.

[0]: dotnet/android@a9f6574
[1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-d15-9/30/API_20Compatibility_20Checks/
[2]: dotnet/android#1836 (comment)
[3]: https://www.nuget.org/packages/System.Memory/
@lassana
Copy link

lassana commented Nov 6, 2018

Hi everyone,
which version of Xamarin.Android includes this?

jonpryor pushed a commit that referenced this pull request Apr 10, 2019
Fixes: xamarin/monodroid#800
Context: xamarin/monodroid@bee5f69

When the Mono team works on a PR to bump mono (e.g. [PR 1536][0]),
they need to deal with two different build systems:

 1. The xamarin-android build system, and
 2. The monodroid build system.

Shocking as this may sound, the OSS xamarin-android build system gets
more love from the Xamarin.Android team than the commercial monodroid
build system.  As such, the monodroid build system is "baroque", not
well tested off of a limited number of machines, prone to peculiar
breakages, and otherwise adds a level of complexity and annoyance
that everyone would rather do without.

Unfortunately, *because* the monodroid repo is private, we can't have
the xamarin-android repo contain a "proper" git submodule to it, as
that would break all external contributors.  Thus, historically, the
monodroid repo instead had a git submodule reference to the
xamarin-android repo, which brings a *different* set of problems,
e.g. the commercial builds are *always* older than the OSS builds,
and accidental breakage is caught much later than we'd like.

The solution?  For commercial builds, instead of "starting with" the
monodroid repo, which submodules xamarin-android and (not-always-
correctly) builds xamarin-android, we instead have the
xamarin-android repo "reference" the monodroid repo, and have "hooks"
in the xamarin-android build system so that monodroid can be built
*from xamarin-android*.  This will simplify life for the Mono team,
allow for (hopefully) easier and more frequent commercial builds,
simplify integration testing of commercial builds, and hopefully
result in fewer apologies from @jonpryor about monodroid builds.

Now, about that "reference": if we can't use git submodules to link
the monodroid repo to the xamarin-android repo, how do we maintain
that reference?


~~ .external ~~

There is a (seldom-used) convention among some Xamarin projects: the
[`.external`][1] file.  This is a plain-text file which allows
specifying external dependencies.  Blank lines and lines beginning
with `#` are ignored, as well as lines which don't match a valid
format.  The external dependency reference format is:

	$(RepoOwner)/$(RepoName):$(BranchName)@$(CommitHash)

We will use the `.external` file in order to link a specific
monodroid commit to the xamarin-android build.

This file *may* be used in the future for mono, as part of the
[Mono Archive][2] work.

The new `make prepare-external-git-dependencies` rule will clone all
git submodule references listed in the `.external` file.  This rule
indirectly uses the new `external-git-dependencies.targets` MSBuild
file, which exposes two new targets:

  * `ParseExternalFile`: Parses `.external` and creates the
    `@(Externals)` item group, which contains item metadata:
      * `%(Owner)`: The GitHub owner/organization
      * `%(Name)`: The git repo name
      * `%(Branch)`: The git repo branch
      * `%(Identity)`: The git commit in the repo

  * `CheckoutExternalGitDependency`: Clones or updates the git repos
     specified in `@(Externals)` into
     `$(ExternalSourceDependencyDirectory)`, which defaults to the
     top-level `external` directory.

The new `<ParseExternalGitDependencies/>` task which is used by the
`ParseExternalFile` target, while the new
`<CheckoutExternalGitDependency/>` task is used by the
`CheckoutExternalGitDependency` target.

`external-git-dependencies.target` is `<Import/>`ed by
`xa-prep-tasks.csproj` to allow us to easily checkout, download, and
process the `.external` file.


~~ Build system hooks ~~

The `make jenkins` target has been turned into a
[double-colon rule][3], which allows the `make jenkins` target to be
specified multiple times, in multiple files, separately.

There is also now an optional [**make** `include`][4] to
`external/monodroid/xa-integration.mk`.
`monodroid/xa-integration.mk` overrides the `make jenkins` target so
that it can integrate it's build into the xamarin-android build.


~~ Installers ~~

`OpenTK.dll` and it's related files have been removed from the
installers, in an effort to start reducing the interdependencies
between the monodroid & xamarin-android repos.

Finally, add a way to override a portion of our installer filenames,
while still using version information from xamarin-android, by
setting `$(UseCommercialInstallerName)`=True.

[0]: #1836
[1]: https://github.com/xamarin/external
[2]: https://github.com/xamarin/xamarin-android/projects/10
[3]: https://www.gnu.org/software/make/manual/html_node/Double_002dColon.html
[4]: https://www.gnu.org/software/make/manual/html_node/Include.html
@github-actions github-actions bot locked and limited conversation to collaborators Feb 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
full-mono-integration-build For PRs; run a full build (~6-10h for mono bumps), not the faster PR subset (~2h for mono bumps)
Projects
None yet
Development

Successfully merging this pull request may close these issues.