New jit intrinsic support

[AndyAyersMS]

Implementation for new-style jit intrinsics, for methods with IL implementations that have known behaviors. The jit can replace these with alternate implementations if it so chooses.

I have been using Enum.HasFlag as a test case, and so have marked it here as [Intrinsic]. The jit currently recognizes the call as a new-style intrinsic but does not yet optimize.

Related issue: #13813.

Plan on doing some cleanup once there is agreement on naming and other aspects of the implementation.

Desktop implementation of the jit interface (not shown) will be stubbed out to always return null values for the new jit interface call.

[AndyAyersMS]

cc @dotnet/jit-contrib

There is always some interface related bit I forget to fix, because we never build it by default anywhere -- anyone remember what that is? [Never mind, found it...]

Will need some up-front desktop changes; also bumps the jit GUID so will need new SPMI data.

[JosephTremoulet]

This might (?) be a good place for a comment indicating that the jit MAY replace direct calls to [Intrinsic] methods with its own sequence, but that it also may opt to fall back on the MSIL implementation (and presumably indirect calls will execute the fall-back implementation?).

[AndyAyersMS]

Added.

[jkotas]

Thanks!

[AndyAyersMS]

Need to lay some groundwork on the desktop side before this can go in.

[AndyAyersMS]

Something about this change seems to be causing the ubuntu and tizen arm test legs to time out. There's nothing platform or arch specific here so it is a bit of a puzzle why they are selectively impacted.

Also seeing the regexdna failure on OSX.

[AndyAyersMS]

Maybe I need to explicitly clear the intrinsic flag in m_bFlags2 for non-intrinsics? If that flag didn't get zero initialized, then all calls would look like intrinsic calls to the jit and it would spend time doing useless name lookups.

[jkotas]

Maybe I need to explicitly clear the intrinsic flag in m_bFlags2 for non-intrinsics?

It should not be needed. All MethodDesc flags are zero-initialized.

[AndyAyersMS]

Hmm, going to try and repro this locally, as those legs don't give much insight as to what they are doing during the test execution phase.

[AndyAyersMS]

As best I can tell it is stuck running the first test:

root@c296aca5efeb:/home# ps -aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  18000  2860 ?        Ss   01:00   0:00 /bin/bash ./tests/scripts/arm32_ci_test.sh --abi=arm --buildConfig=Release
root        14  0.0  0.1 4137488 10496 ?       S    01:00   0:00 /usr/bin/qemu-arm-static /bin/bash -x
root        15  0.0  0.1 4137488 12208 ?       S    01:00   0:00 /usr/bin/qemu-arm-static /bin/bash ./tests/runtest.sh --sequential --coreClrBinDir=/home/coreclr/bin/Product/Linux.arm.Release --mscorlibDir
root        60  0.0  0.1 4137488 10600 ?       S    01:00   0:00 /usr/bin/qemu-arm-static /bin/bash ./tests/runtest.sh --sequential --coreClrBinDir=/home/coreclr/bin/Product/Linux.arm.Release --mscorlibDir
root        69  0.0  0.1 4137488 10524 ?       S    01:00   0:00 /usr/bin/qemu-arm-static /bin/bash ./ldelemnullarr2.sh
root        73 99.9  0.4 4141880 32716 ?       RLl  01:00  13:17 /usr/bin/qemu-arm-static /home/coreclr/bin/tests/Windows_NT.x64.Release/Tests/coreoverlay/corerun ldelemnullarr2.exe
root        82  0.0  0.0  18140  3248 ?        Ss   01:13   0:00 /bin/bash
root        99  0.0  0.0  15568  2136 ?        R+   01:14   0:00 ps -aux

[AndyAyersMS]

I can't get a user mode debugger attached yet, but I enabled system call tracing in qemu and it looks like the process is stuck in some kind of waiting pattern:

75 futex(0x000736b0,FUTEX_PRIVATE_FLAG|FUTEX_WAKE,1,NULL,NULL,0) = 0
75 clock_gettime(1,-232862952,-232862900,-232862904,472344,10000) = 0
75 futex(0x000736cc,FUTEX_PRIVATE_FLAG|FUTEX_WAIT_BITSET,7,0xf21ecb18,NULL,0) = -1 errno=110 (Connection timed out)
75 futex(0x000736b0,FUTEX_PRIVATE_FLAG|FUTEX_WAKE,1,NULL,NULL,0) = 0
75 clock_gettime(1,-232862952,-232862900,-232862904,472344,10000) = 0
75 futex(0x000736cc,FUTEX_PRIVATE_FLAG|FUTEX_WAIT_BITSET,9,0xf21ecb18,NULL,0) = -1 errno=110 (Connection timed out)
75 futex(0x000736b0,FUTEX_PRIVATE_FLAG|FUTEX_WAKE,1,NULL,NULL,0) = 0
75 clock_gettime(1,-232862952,-232862900,-232862904,472344,10000) = 0
75 futex(0x000736cc,FUTEX_PRIVATE_FLAG|FUTEX_WAIT_BITSET,11,0xf21ecb18,NULL,0) = -1 errno=110 (Connection timed out)
...

@dotnet/arm32-contrib @janvorli any ideas what might be going wrong here?

Any tips on how to debug corerun running under qemu inside docker on ubuntu?

[janvorli]

@AndyAyersMS as for debugging inside qemu, I think that you should be run gdb or at least gdbserver inside qemu. But maybe the easiest way is to borrow a Raspberry PI 2 or 3 and debug the stuff on it.

You may also be able to use the new createdump tool that's built in the coreclr repo. According to @mikem8361, it should be able to create a dump of a live process.

[AndyAyersMS]

Using a checked runtime gives the following clue:

               Assert failure(PID 803 [0x00000323], Thread: 803 [0x0323]): Reached the "unreachable": This is a JIT intrinsic!FAILED: <unreachable>
                   File: /opt/code/src/vm/stubhelpers.cpp Line: 1888
                   Image: /home/coreclr/bin/tests/Windows_NT.x64.Release/Tests/coreoverlay/corerun

So maybe something in the jit intrinsic handling gets messed up. Still digging...

[AndyAyersMS]

The assert fires after the first jitted method is run (no crossgen, so this is System.AppDomain::SetupDomain). With this change the jit does not attempt to process any intrinsics in that method (haven't checked what happens without the change yet).

Removing the second if from the jit interface change "fixes" things:

    if (pMD->IsFCallOrIntrinsic())
        result |= CORINFO_FLG_NOGCCHECK | CORINFO_FLG_INTRINSIC;
    if (pMD->IsJitIntrinsic())
       result |= CORINFO_FLG_JIT_INTRINSIC | CORINFO_FLG_INTRINSIC;

So I wonder if maybe this is a compiler bug, and because of it, we end up not setting the intrinsic flag when we should? Am going to add an assert at the end to see....

[AndyAyersMS]

Above seems more plausible now -- adding

    if (pMD->IsFCallOrIntrinsic())
     {
       _ASSERTE((result & CORINFO_FLG_INTRINSIC) != 0);
     }

to the end of getMethodAttribsInternal and it fires.

[AndyAyersMS]

Revised the code so that we no longer set CORINFO_FLG_INTRINSIC for the new intrinsics. Instead we look for both intrinsic flags on the jit side.

[AndyAyersMS]

Hmm, looks like some legs are hitting an infrastructure issue:

ERROR: Error cloning remote repo 'origin'
11:09:14 hudson.plugins.git.GitException: Command "git fetch --tags --progress https://github.com/dotnet/coreclr.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
11:09:14 stdout: 
11:09:14 stderr: fatal: unable to access 'https://github.com/dotnet/coreclr.git/': error setting certificate verify locations:
11:09:14   CAfile: C:/Program Files/Git/mingw64/libexec/ssl/certs/ca-bundle.crt
11:09:14   CApath: none

[AndyAyersMS]

@dotnet/dnceng some connectivity issues with the perf correctness legs since about 9AM.

[mmitche]

AGH.....I had uninstalled git and reinstalled it....why is it having cert issues.

[AndyAyersMS]

Looks like maybe the perf correctness issues are fixed ;-- wii retry.
@dotnet-bot retest Windows_NT x64 full_opt ryujit CoreCLR Perf Tests Correctness
@dotnet-bot retest Windows_NT x86 full_opt legacy_backend CoreCLR Perf Tests Correctness
@dotnet-bot retest Windows_NT x86 full_opt ryujit CoreCLR Perf Tests Correctness

[fiigii]

Jit intrinsics are always optional to expand

This will not work for the new SIMD intrinsics. We can adjust it once we get there.

@AndyAyersMS @jkotas Could you please explain this more? I am using the new JIT intrinsic recognizer to implement Intel hardware intrinsics.

[jkotas]

The hardware intrinsics has to be always expanded unconditionally, even when optimizations are off.

[fiigii]

even when optimizations are off.

So the new jit intrinsic expansion depends on optimization config?

[jkotas]

Right. It is how the intrinsic expansion has been historically always wired in the JIT - it was turned on only when optimizations are on.

[fiigii]

@jkotas Thanks. I think we should resolve this now. I have implemented IsSupported, but the returned const node cannot be expanded. Could you give me suggestions for the solution?

[jkotas]

I would defer to @dotnet/jit-contrib on details how the hardware intrinsics expansion should be wired in the JIT. Maybe you can open WIP PR with what you have done for IsSupported and the details can be discussed there.

[fiigii]

Maybe you can open WIP PR with what you have done for IsSupported and the details can be discussed there.

Will do.