Can't build runtime on Ubuntu 20.04 (LTS) due to native PGO format

[EgorBo]

I followed the official steps to configure my Ubuntu 20.04 and when I try to build the runtime it fails with

  error: Could not read profile /home/egorbo/.nuget/packages/optimization.linux-x64.pgo.coreclr/1.0.0-
prerelease.23260.3/data/coreclr.profdata: Unsupported instrumentation profile format version
  1 error generated.

(and builds fine with /p:NoPgoOptimize=true)

Is it expected? My clang --version is 10.0.0-4ubuntu did we bump minimal requirement to 16?

@sbomer (cc @directhex who might be familiar with it too)
Also, cc @sebastienros in case if he hit a similar build issue for TE. The problem that we noticed a serios regression in TE instrumented-tiers benchmark which might be affected by Native PGO (or lack of it)

PS: I use ubuntu 20.04 because it uses the correct libc for TE benchmarks, I believe I can't update to 22

cc @AndyAyersMS

[sbomer]

Currently, PGO optimization requires clang-12 on linux because we are consuming PGO data that was collected from clang-12-instrumented runtime bits. That will very soon change to clang-16 once we round-trip the clang-16 instrumented PGO data (https://dev.azure.com/dnceng/internal/_build/results?buildId=2184487&view=results).

I'm working on turning off PGO by default in local builds to bring things back into alignment with the official prereqs (see #85785). I'll make sure we don't flow the new PGO data until that is done, but if you want PGO optimization, you'll need clang-16 soon. One way to do that is to use the mariner images: https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/linux-instructions.md#docker-images.

[EgorBo]

We believe that broken native PGO is responsible for these huge regression in TE:

PGO page, instrumented tiers

[sbomer]

I wonder if it could be the result of using an older version of the profile data format (whatever was used in clang-12) to optimize with clang-16. The timing looks like it roughly lines up with #84676. If that's the case, then I would also expect it to be fixed when we ingest the new clang-16 profile data.

[sebastienros]

@kunalspathak probably what I mentioned earlier for Ampere machines.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I followed the official steps to configure my Ubuntu 20.04 and when I try to build the runtime it fails with

  error: Could not read profile /home/egorbo/.nuget/packages/optimization.linux-x64.pgo.coreclr/1.0.0-
prerelease.23260.3/data/coreclr.profdata: Unsupported instrumentation profile format version
  1 error generated.

(and builds fine with /p:NoPgoOptimize=true)

Is it expected? My clang --version is 10.0.0-4ubuntu did we bump minimal requirement to 16?

@sbomer (cc @directhex who might be familiar with it too)
Also, cc @sebastienros in case if he hit a similar build issue for TE. The problem that we noticed a serios regression in TE instrumented-tiers benchmark which might be affected by Native PGO (or lack of it)

PS: I use ubuntu 20.04 because it uses the correct libc for TE benchmarks, I believe I can't update to 22

cc @AndyAyersMS

Author:	EgorBo
Assignees:	-
Labels:	area-Infrastructure, untriaged, needs-area-label
Milestone:	-

[sbomer]

The build failure should be fixed by #86436. Can we close this, or do we need it as a tracking issue for the perf regression?

[EgorBo]

Thanks! Yes, let's close it, thanks for the clarification.