Unable to create profile with create_llvm_prof - Empty profile map error

[cristianassaiante]

Hi, I'm trying to compile clang using AutoFDO. In order, I'm running the following (adapted from the propeller script):

• Build baseline clang to generate profiles

COMMON_CMAKE_FLAGS=(
    "-DLLVM_OPTIMIZED_TABLEGEN=On"
    "-DCMAKE_BUILD_TYPE=RelWithDebInfo"
    "-DLLVM_TARGETS_TO_BUILD=X86"
    "-DLLVM_ENABLE_PROJECTS=clang"
    "-DCMAKE_C_COMPILER=${PATH_TO_TRUNK_LLVM_BUILD}/bin/clang"
    "-DCMAKE_CXX_COMPILER=${PATH_TO_TRUNK_LLVM_BUILD}/bin/clang++"
    "-DLLVM_BUILD_INSTRUMENTED=ON" )

cmake -G Ninja "${COMMON_CMAKE_FLAGS[@]}"\
    ${PATH_TO_SOURCES}/llvm-project/llvm
ninja -j 40 clang

• Use perf to obtain branch data

ninja -t commands | head -100 >& ./perf_commands.sh
chmod +x ./perf_commands.sh
perf record -j any,u ./perf_commands.sh

• Use create_llvm_prof to generate the profile
  create_llvm_prof --binary $PATH_TO_BINARIES/clang.baseline --out baseline.prof

Everything succeeds, until the very last command which gives the following output:

[WARNING:/home/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1322] Skipping 492 bytes of metadata: HEADER_CPU_TOPOLOGY
[WARNING:autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1069] Skipping unsupported event PERF_RECORD_CPU_MAP
[INFO:/home/autofdo/third_party/perf_data_converter/src/quipper/perf_reader.cc:1060] Number of events stored: 8366776
[INFO:/home/autofdo/third_party/perf_data_converter/src/quipper/perf_parser.cc:272] Parser processed: 1221 MMAP/MMAP2 events, 112 COMM events, 110 FORK events, 111 EXIT events, 8336706 SAMPLE events, 8287979 of these were mapped, 0 SAMPLE events with a data address, 0 of these were mapped
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240926 10:32:30.280131 89010 sample_reader.cc:289] No buildid found in binary
I20240926 10:32:34.213464 89010 symbol_map.cc:477] Adding loadable exec segment: offset=1597000 vaddr=1597000
W20240926 10:32:42.118443 89010 llvm_profile_writer.cc:50] Got an empty profile map. The output file might still be not empty (e.g., containing symbol list in binary format) but might be not helpful as a profile

I have tested several configurations and also tried to use different perf option, but the outcome does not seems to change.

Thanks in advance.

[snehasish]

Can you drop the "DLLVM_BUILD_INSTRUMENTED=ON" flag? That shouldn't be necessary.

Can you confirm that clang.baseline is the one obtained from the build step above? Also check that the binary was built with debug info.

cc: @shenhanc78 can you take a look? Any ideas / tips to debug further? Thanks

[snehasish]

Looking at the error log again, since you are using a script to invoke perf multiple times, I think a build id is necessary to aggregate the samples correctly. Can you try building clang itself with an additional linker flag -Wl,--build-id=sha1?

[shenhanc78]

Yes, buildid is required. You may also use --ignore_build_id to skip this test. So all "clang" binary events (instead of only clang binary with the same buildid as recorded in perf.data file) in the perf will be processed.

[cristianassaiante]

Thank you so much for your support!
Adding the linker flag to generate the build-id made the job.

[cristianassaiante]

@snehasish after generating the profile, I'm feeding it through cmake using -DLLVM_PROFDATA_FILE, but unfortunately clang exits with the following error:

Error in reading profile [...]/clang.prof: invalid instrumentation profile data (bad magic)

I've tried every format among those supported by create_llvm_prof, but it doesn't seem to work.
I'm using LLVM 18.0.0 (commit-id: 05c16f4).

The first 10 lines of clang.prof in text format are the following:

_ZN4llvm3MD54bodyENS_8ArrayRefIhEE:16578301:68
 0: 46
 8: 46
 9: 46
 10: 46
 11: 46
 20: 7331
 21: 7331
 22: 7331
 23: 7331

[snehasish]

-DLLVM_PROFDATA_FILE is meant to pass the instrumentation profile and thus it maps to the -fprofile-instr-use flag. For AutoFDO you need the -fprofile-sample-use flag (https://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers). There is no CMake option for AutoFDO.

Perhaps changing the option here [1] will be enough to pass the right flag to the compiler and linker.

[1] https://github.com/llvm/llvm-project/blob/e565a4fa0b09456265e28b017054b20ff4315c58/llvm/cmake/modules/HandleLLVMOptions.cmake#L1213

[cristianassaiante]

Thank you so much!