Skip to content

Commit

Permalink
Implement support for UnsafeAccessor in the trimmer (#88268)
Browse files Browse the repository at this point in the history
The core of the change is that `UnsafeAccessor` creates a code dependency from the accessor method to the target specified by the attribute. The trimmer needs to follow this dependency and preserve the target. Additionally, because the trimmer operates at the IL level, it needs to make sure that the target will keep its name and some other properties intact (so that the runtime implementation of the `UnsafeAccessor` can still work).

Implementation choices:
* The trimmer will mark the target as "accessed via reflection", this is a simple way to make sure that name and other properties about the target are preserved. This could be optimized in the future, but the savings are probably not that interesting.
* The implementation ran into a problem when trying to precisely match the signature overload resolution. Due to Cecil issues and the fact that Cecil's resolution algorithm is not extensible, it was not possible to match the runtime's behavior without adding lot more complexity (currently it seems we would have to reimplement method resolution in the trimmer). So, to simplify the implementation, trimmer will mark all methods of a given name. This means it will mark more than necessary. This is fixable by adding more complexity to the code base if we think there's a good reason for it.
* Due to the above choices, there are some behavioral differences:
  * Trimmer will warn if the target has data flow annotations, always. There's no way to "fix" this in the code without a suppression.
  * Trimmer will produce different warning codes even if there is a true data flow mismatch - this is because it treats the access as "reflection access" which produces different warning codes from direct access.
  * These differences are fixable, but it was not deemed necessary right now.
* We decided that analyzer will not react to the attribute at all, and thus will not produce any diagnostics around it.

The guiding reason to keep the implementation simple is that we don't expect the unsafe accessor to be used by developers directly, instead we assume that vast majority of its usages will be from source generators. So, developer UX is not as important.

Test changes:
* Adds directed tests for the marking behavior
* Adds tests to verify that `Requires*` attributes behave correctly
* Adds tests to verify that data flow annotations behave as expected (described above)
* The tests are effectively a second validation of the NativeAOT implementation as they cover NativeAOT as well.

Fixes in CoreCLR/NativeAOT:
This change fixes one bug in the CoreCLR/NativeAOT implementation, unsafe accessor on a instance method of a value type must use "by-ref" parameter for the `this` parameter. Without the "by-ref" the accessor is considered invalid and will throw.
This change also adds some tests to the CoreCLR/NativeAOT test suite.

Part of #86161.
Related to #86438.
Feature design in #81741.
  • Loading branch information
vitek-karas authored Jul 18, 2023
1 parent 595cbd1 commit ac3979a
Show file tree
Hide file tree
Showing 13 changed files with 1,345 additions and 2 deletions.
2 changes: 1 addition & 1 deletion eng/pipelines/runtime-linker-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ extends:
- linux_x64
jobParameters:
testGroup: innerloop
testResultsFormat: 'vstest'
enablePublishTestResults: true
timeoutInMinutes: 120
nameSuffix: ILLink_Runtime_Testing
condition:
Expand Down
9 changes: 9 additions & 0 deletions src/coreclr/tools/Common/TypeSystem/IL/UnsafeAccessors.cs
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,15 @@ public static MethodIL TryGetIL(EcmaMethod method)
return GenerateAccessorBadImageFailure(method);
}

// If the non-static method access is for a
// value type, the instance must be byref.
if (kind == UnsafeAccessorKind.Method
&& firstArgType.IsValueType
&& !firstArgType.IsByRef)
{
return GenerateAccessorBadImageFailure(method);
}

context.TargetType = ValidateTargetType(firstArgType);
if (context.TargetType == null)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ public void Reflection (string t)
switch (t) {
case "TypeHierarchyReflectionWarnings":
case "ParametersUsedViaReflection":
case "UnsafeAccessor":
Run (t);
break;
default:
Expand Down
9 changes: 9 additions & 0 deletions src/coreclr/vm/prestub.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1503,6 +1503,15 @@ bool MethodDesc::TryGenerateUnsafeAccessor(DynamicResolver** resolver, COR_ILMET
if (firstArgType.IsNull())
ThrowHR(COR_E_BADIMAGEFORMAT, BFA_INVALID_UNSAFEACCESSOR);

// If the non-static method access is for a
// value type, the instance must be byref.
if (kind == UnsafeAccessorKind::Method
&& firstArgType.IsValueType()
&& !firstArgType.IsByRef())
{
ThrowHR(COR_E_BADIMAGEFORMAT, BFA_INVALID_UNSAFEACCESSOR);
}

context.TargetType = ValidateTargetType(firstArgType);
context.IsTargetStatic = kind == UnsafeAccessorKind::StaticMethod;
if (!TrySetTargetMethod(context, name.GetUTF8()))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ struct UserDataValue

private static string _M(string s, ref string sr, in string si) => s;
private string _m(string s, ref string sr, in string si) => s;

public string GetFieldValue() => _f;
}

[UnsafeAccessor(UnsafeAccessorKind.Constructor)]
Expand Down Expand Up @@ -119,6 +121,18 @@ public static void Verify_CallCtorValue()
Assert.Equal(PrivateArg, local.Value);
}

[Fact]
public static void Verify_CallCtorWithEmptyNotNullName()
{
Console.WriteLine($"Running {nameof(Verify_CallCtorWithEmptyNotNullName)}");

var local = CallPrivateConstructorWithEmptyName();
Assert.Equal(nameof(UserDataClass), local.GetType().Name);

[UnsafeAccessor(UnsafeAccessorKind.Constructor, Name="")]
extern static UserDataClass CallPrivateConstructorWithEmptyName();
}

[Fact]
public static void Verify_CallCtorAsMethod()
{
Expand Down Expand Up @@ -191,6 +205,10 @@ public static void Verify_AccessFieldValue()
UserDataValue local = new();
Assert.Equal(Private, GetPrivateField(ref local));

const string newValue = "__NewValue__";
GetPrivateField(ref local) = newValue;
Assert.Equal(newValue, local.GetFieldValue());

[UnsafeAccessor(UnsafeAccessorKind.Field, Name=UserDataValue.FieldName)]
extern static ref string GetPrivateField(ref UserDataValue d);
}
Expand Down Expand Up @@ -355,6 +373,50 @@ public static void Verify_ManagedUnmanagedFunctionPointersDontMatch()
extern static string CallManagedMethod(UserDataClass d, delegate* unmanaged[Cdecl]<void> fptr);
}

class InheritanceBase
{
private static string OnBase() => nameof(OnBase);
private static string FieldOnBase = nameof(FieldOnBase);
}

class InheritanceDerived : InheritanceBase
{
private static string OnDerived() => nameof(OnDerived);
private static string FieldOnDerived = nameof(FieldOnDerived);
}

[Fact]
public static void Verify_InheritanceMethodResolution()
{
Console.WriteLine($"Running {nameof(Verify_InheritanceMethodResolution)}");

var instance = new InheritanceDerived();
Assert.Throws<MissingMethodException>(() => OnBase(instance));
Assert.Equal(nameof(OnDerived), OnDerived(instance));

[UnsafeAccessor(UnsafeAccessorKind.StaticMethod, Name = nameof(OnBase))]
extern static string OnBase(InheritanceDerived i);

[UnsafeAccessor(UnsafeAccessorKind.StaticMethod, Name = nameof(OnDerived))]
extern static string OnDerived(InheritanceDerived i);
}

[Fact]
public static void Verify_InheritanceFieldResolution()
{
Console.WriteLine($"Running {nameof(Verify_InheritanceFieldResolution)}");

var instance = new InheritanceDerived();
Assert.Throws<MissingFieldException>(() => FieldOnBase(instance));
Assert.Equal(nameof(FieldOnDerived), FieldOnDerived(instance));

[UnsafeAccessor(UnsafeAccessorKind.StaticField, Name = nameof(FieldOnBase))]
extern static ref string FieldOnBase(InheritanceDerived i);

[UnsafeAccessor(UnsafeAccessorKind.StaticField, Name = nameof(FieldOnDerived))]
extern static ref string FieldOnDerived(InheritanceDerived i);
}

[Fact]
public static void Verify_InvalidTargetUnsafeAccessor()
{
Expand Down Expand Up @@ -389,6 +451,13 @@ public static void Verify_InvalidTargetUnsafeAccessor()
AssertExtensions.ThrowsMissingMemberException<MissingMethodException>(
isNativeAot ? null : UserDataClass.MethodPointerName,
() => CallPointerMethod(null, null));
AssertExtensions.ThrowsMissingMemberException<MissingMethodException>(
isNativeAot ? null : UserDataClass.StaticMethodName,
() => { string sr = string.Empty; StaticMethodWithDifferentReturnType(null, null, ref sr, string.Empty); });

AssertExtensions.ThrowsMissingMemberException<MissingMethodException>(
isNativeAot ? null : UserDataClass.StaticMethodName,
() => { string sr = string.Empty; StaticMethodWithDifferentReturnType(null, null, ref sr, string.Empty); });

[UnsafeAccessor(UnsafeAccessorKind.Method, Name=DoesNotExist)]
extern static void MethodNotFound(UserDataClass d);
Expand All @@ -412,6 +481,8 @@ public static void Verify_InvalidTargetUnsafeAccessor()
[UnsafeAccessor(UnsafeAccessorKind.StaticMethod, Name=UserDataClass.MethodPointerName)]
extern static string CallPointerMethod(UserDataClass d, delegate* unmanaged[Stdcall]<void> fptr);

[UnsafeAccessor(UnsafeAccessorKind.StaticMethod, Name=UserDataClass.StaticMethodName)]
extern static int StaticMethodWithDifferentReturnType(UserDataClass d, string s, ref string sr, in string si);
}

[Fact]
Expand Down Expand Up @@ -467,6 +538,12 @@ public static void Verify_InvalidUseUnsafeAccessor()
Assert.Throws<BadImageFormatException>(() => new Invalid().NonStatic(string.Empty));
Assert.Throws<BadImageFormatException>(() => Invalid.CallToString<string>(string.Empty));
Assert.Throws<BadImageFormatException>(() => Invalid<string>.CallToString(string.Empty));
Assert.Throws<BadImageFormatException>(() =>
{
string str = string.Empty;
UserDataValue local = new();
InvokeMethodOnValueWithoutRef(local, null, ref str, str);
});

[UnsafeAccessor(UnsafeAccessorKind.Field, Name=UserDataValue.FieldName)]
extern static string FieldReturnMustBeByRefClass(UserDataClass d);
Expand Down Expand Up @@ -503,5 +580,8 @@ public static void Verify_InvalidUseUnsafeAccessor()

[UnsafeAccessor(UnsafeAccessorKind.Method, Name=nameof(ToString))]
extern static string LookUpFailsOnFunctionPointers(delegate* <void> fptr);

[UnsafeAccessor(UnsafeAccessorKind.Method, Name = UserDataValue.MethodName)]
extern static string InvokeMethodOnValueWithoutRef(UserDataValue target, string s, ref string sr, in string si);
}
}
13 changes: 13 additions & 0 deletions src/tools/illink/src/linker/Linker.Steps/MarkStep.cs
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Reflection.Runtime.TypeParsing;
using System.Runtime.CompilerServices;
using System.Text.RegularExpressions;
using ILLink.Shared;
using ILLink.Shared.TrimAnalysis;
Expand Down Expand Up @@ -127,6 +128,7 @@ internal DynamicallyAccessedMembersTypeHierarchy DynamicallyAccessedMembersTypeH
DependencyKind.ReferencedBySpecialAttribute,
DependencyKind.TypePreserve,
DependencyKind.XmlDescriptor,
DependencyKind.UnsafeAccessorTarget,
};

static readonly DependencyKind[] _typeReasons = new DependencyKind[] {
Expand Down Expand Up @@ -211,6 +213,7 @@ internal DynamicallyAccessedMembersTypeHierarchy DynamicallyAccessedMembersTypeH
DependencyKind.FieldMarshalSpec,
DependencyKind.ReturnTypeMarshalSpec,
DependencyKind.XmlDescriptor,
DependencyKind.UnsafeAccessorTarget,
};
#endif

Expand Down Expand Up @@ -1870,6 +1873,7 @@ void ProcessAnalysisAnnotationsForField (FieldDefinition field, DependencyKind d
case DependencyKind.DynamicallyAccessedMember:
case DependencyKind.InteropMethodDependency:
case DependencyKind.Ldtoken:
case DependencyKind.UnsafeAccessorTarget:
if (isReflectionAccessCoveredByDAM = Annotations.FlowAnnotations.ShouldWarnWhenAccessedForReflection (field))
Context.LogWarning (origin, DiagnosticId.DynamicallyAccessedMembersFieldAccessedViaReflection, field.GetDisplayName ());

Expand Down Expand Up @@ -3264,6 +3268,10 @@ protected virtual void ProcessMethod (MethodDefinition method, in DependencyInfo
ProcessInteropMethod (method);
}

if (!method.HasBody || method.Body.CodeSize == 0) {
ProcessUnsafeAccessorMethod (method);
}

if (ShouldParseMethodBody (method))
MarkMethodBody (method.Body);

Expand Down Expand Up @@ -3494,6 +3502,11 @@ void ProcessInteropMethod (MethodDefinition method)
#pragma warning restore RS0030
}

void ProcessUnsafeAccessorMethod (MethodDefinition method)
{
(new UnsafeAccessorMarker (Context, this)).ProcessUnsafeAccessorMethod (method);
}

protected virtual bool ShouldParseMethodBody (MethodDefinition method)
{
if (!method.HasBody)
Expand Down
141 changes: 141 additions & 0 deletions src/tools/illink/src/linker/Linker.Steps/UnsafeAccessorMarker.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
// Copyright (c) .NET Foundation and contributors. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.Runtime.CompilerServices;
using Mono.Cecil;

namespace Mono.Linker.Steps
{
// This class only handles static methods (all the unsafe accessors should be static)
// so there's no problem with forgetting the implicit "this".
#pragma warning disable RS0030 // MethodReference.Parameters is banned

readonly struct UnsafeAccessorMarker (LinkContext context, MarkStep markStep)
{
readonly LinkContext _context = context;
readonly MarkStep _markStep = markStep;

// We don't perform method overload resolution based on list of parameters (or return type) for now
// Mono.Cecil's method resolution is problematic and has bugs. It's also not extensible
// and we would need that to correctly implement the desired behavior around custom modifiers. So for now we decided to not
// duplicate the logic to tweak it and will just mark entire method groups.

public void ProcessUnsafeAccessorMethod (MethodDefinition method)
{
if (!method.IsStatic || !method.HasCustomAttributes)
return;

foreach (CustomAttribute customAttribute in method.CustomAttributes) {
if (customAttribute.Constructor.DeclaringType.FullName == "System.Runtime.CompilerServices.UnsafeAccessorAttribute") {
if (customAttribute.HasConstructorArguments && customAttribute.ConstructorArguments[0].Value is int kindValue) {
UnsafeAccessorKind kind = (UnsafeAccessorKind) kindValue;
string? name = null;
if (customAttribute.HasProperties) {
foreach (CustomAttributeNamedArgument prop in customAttribute.Properties) {
if (prop.Name == "Name") {
name = prop.Argument.Value as string;
break;
}
}
}

switch (kind) {
case UnsafeAccessorKind.Constructor:
ProcessConstructorAccessor (method, name);
break;
case UnsafeAccessorKind.StaticMethod:
ProcessMethodAccessor (method, name, isStatic: true);
break;
case UnsafeAccessorKind.Method:
ProcessMethodAccessor (method, name, isStatic: false);
break;
case UnsafeAccessorKind.StaticField:
ProcessFieldAccessor (method, name, isStatic: true);
break;
case UnsafeAccessorKind.Field:
ProcessFieldAccessor (method, name, isStatic: false);
break;
default:
break;
}

// Intentionally only process the first such attribute
// if there's more than one runtime will fail on it anyway.
break;
}
}
}
}

void ProcessConstructorAccessor (MethodDefinition method, string? name)
{
// A return type is required for a constructor, otherwise
// we don't know the type to construct.
// Types should not be parameterized (that is, by-ref).
// The name is defined by the runtime and should be empty.
if (method.ReturnsVoid () || method.ReturnType.IsByRefOrPointer () || !string.IsNullOrEmpty (name))
return;

if (_context.TryResolve (method.ReturnType) is not TypeDefinition targetType)
return;

foreach (MethodDefinition targetMethod in targetType.Methods) {
if (!targetMethod.IsConstructor || targetMethod.IsStatic)
continue;

_markStep.MarkMethodVisibleToReflection (targetMethod, new DependencyInfo (DependencyKind.UnsafeAccessorTarget, method), new MessageOrigin (method));
}
}

void ProcessMethodAccessor (MethodDefinition method, string? name, bool isStatic)
{
// Method access requires a target type.
if (method.Parameters.Count == 0)
return;

if (string.IsNullOrEmpty (name))
name = method.Name;

TypeReference targetTypeReference = method.Parameters[0].ParameterType;
if (_context.TryResolve (targetTypeReference) is not TypeDefinition targetType)
return;

if (!isStatic && targetType.IsValueType && !targetTypeReference.IsByReference)
return;

foreach (MethodDefinition targetMethod in targetType.Methods) {
if (targetMethod.Name != name || targetMethod.IsStatic != isStatic)
continue;

_markStep.MarkMethodVisibleToReflection (targetMethod, new DependencyInfo (DependencyKind.UnsafeAccessorTarget, method), new MessageOrigin (method));
}
}

void ProcessFieldAccessor (MethodDefinition method, string? name, bool isStatic)
{
// Field access requires exactly one parameter
if (method.Parameters.Count != 1)
return;

if (string.IsNullOrEmpty (name))
name = method.Name;

if (!method.ReturnType.IsByReference)
return;

TypeReference targetTypeReference = method.Parameters[0].ParameterType;
if (_context.TryResolve (targetTypeReference) is not TypeDefinition targetType)
return;

if (!isStatic && targetType.IsValueType && !targetTypeReference.IsByReference)
return;

foreach (FieldDefinition targetField in targetType.Fields) {
if (targetField.Name != name || targetField.IsStatic != isStatic)
continue;

_markStep.MarkFieldVisibleToReflection (targetField, new DependencyInfo (DependencyKind.UnsafeAccessorTarget, method), new MessageOrigin (method));
}
}
}
}
2 changes: 2 additions & 0 deletions src/tools/illink/src/linker/Linker/DependencyInfo.cs
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,8 @@ public enum DependencyKind
PreservedOperator = 87, // operator method preserved on a type

DynamicallyAccessedMemberOnType = 88, // type with DynamicallyAccessedMembers annotations (including those inherited from base types and interfaces)

UnsafeAccessorTarget = 89, // the member is referenced via UnsafeAccessor attribute
}

public readonly struct DependencyInfo : IEquatable<DependencyInfo>
Expand Down
Loading

0 comments on commit ac3979a

Please sign in to comment.