NetBSD mmap()

[thomasvaughan]

Previously reported to the Monero project as Issue #6592:

Compiling on NetBSD 9 (x86_64) fails with this message:

/home/t/usr/src/RandomX/src/virtual_memory.cpp: In function ‘void* allocLargePagesMemory(std::size_t)’:
/home/t/usr/src/RandomX/src/virtual_memory.cpp:147:83: error: ‘MAP_HUGETLB’ was not declared in this scope
  mem = mmap(nullptr, bytes, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_POPULATE, -1, 0);
                                                                                   ^~~~~~~~~~~

Compiling succeeds if the following is added immediately below the "//OpenBSD does not support huge pages" comment in src/virtualmemory.cpp

#elif defined(__NetBSD__)
    #define RESERVED_FLAGS PROT_MPROTECT(PROT_EXEC)
    mem = mmap(nullptr, bytes, PROT_READ | PROT_WRITE | RESERVED_FLAGS, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);

(and also a && !defined(__NetBSD__) is appended to the end of the #elif !defined(__OpenBSD__) line in src/tests/affinity.cpp). This is a quick cut-and-paste from elsewhere, since I don't know C++, and I'm not suggesting it's sensible.

Then, running randomx-tests gives the following output:
[ 1] Cache initialization ... PASSED
[ 2] SuperscalarHash generator ... PASSED
[ 3] randomx_reciprocal ... PASSED
[ 4] randomx_reciprocal_fast ... PASSED
[ 5] Dataset initialization (interpreter) ... PASSED
[ 6] Dataset initialization (compiler) ... terminate called after throwing an instance of 'std::runtime_error'
what(): mprotect failed
Abort (core dumped)

[tevador]

You put the mmap fix in the wrong place.

Can you test #187?

[thomasvaughan]

With #187, I still see an error upon running randomx-tests:

[ 6] Dataset initialization (compiler)        ... terminate called after throwing an instance of 'std::runtime_error'
  what():  mprotect failed
Abort (core dumped)

Edit: Not sure if it's relevant, but my cut-and paste came from monero-project/monero#5192 (if I remember correctly).
Edit2: Or from monero-project/monero#5251 ?

[tevador]

@thomasvaughan Can you try #187 again? I missed a few places where the W^X policy had to be enabled in the code.

[thomasvaughan]

In case it helps, more verbose output is generated by running paxctl +m && ./randomx-tests. A whole series of tests yield a "PASSED" message, and the last four lines are:

[88] Hash test 2e (compiler)                  ... PASSED
[89] Cache initialization: SSSE3              ... PASSED
[90] Cache initialization: AVX2               ... SKIPPED
[91] Hash batch test

Edit: I wrote the above before I saw your comment above. I'll try again.

[thomasvaughan]

./randomx-tests yields 89 "PASSED" messages, followed by 1 "SKIPPED" message and then a "Memory fault". Here are the last four lines:

[88] Hash test 2e (compiler)                  ... PASSED
[89] Cache initialization: SSSE3              ... PASSED
[90] Cache initialization: AVX2               ... SKIPPED
[91] Hash batch test                          ... Memory fault (core dumped)

(This is without any paxctl +m ./randomx-tests.)

[thomasvaughan]

I may have misrepresented the syntax of the paxctl command in my comments above. To be clear, the manpage says it's possible to disable PaX MPROTECT bby means of paxctl -M ./randomx-tests. Doing so gives more verbose output for test 91:

[91] Hash batch test                          ... assertion "equalsHex(hash1, "639183aae1bf4c9a35884cb46b09cad9175f04efd7684e7262a0ac1c2f0b4e3f")" failed: file "/home/t/usr/src/RandomX/src/tests/tests.cpp", line 1072, function "main()::<lambda()>"
Abort (core dumped)

[tevador]

Can you check with gdb where exactly it crashes in test 91? I mean without disabling W^X.

I'm guessing ./randomx-benchmark --verify --auto will also crash?

[thomasvaughan]

Here's the output from ./randomx-benchmark --verify --auto. No crash messages!

RandomX benchmark v1.1.7
 - Argon2 implementation: SSSE3
 - light memory mode (256 MiB)
 - JIT compiled mode (secure)
 - hardware AES mode
 - small pages mode
Initializing ...
Memory initialized in 0.741305 s
Initializing 1 virtual machine(s) ...
Running benchmark (1000 nonces) ...
Calculated result: 10b649a3f15c7c7f88277812f2e74b337a0f20ce909af09199cccb960771cfa1
Reference result:  10b649a3f15c7c7f88277812f2e74b337a0f20ce909af09199cccb960771cfa1
Performance: 19.36 ms per hash

I'm not fluent in the use of gdb, but entering gdb ./randomx-tests followed by run at the gdb prompt, I see

[91] Hash batch test                          ... 
Program received signal SIGSEGV, Segmentation fault.
0x000000000040d7f0 in randomx::Blake2Generator::checkData(unsigned long) ()

That's with all PaX flags enabled . Incidentally, gdb says (No debugging symbols found in randomx-tests) - does that mean I should recompile somehow if this is to be helpful?

[tevador]

x000000000040d7f0 in randomx::Blake2Generator::checkData(unsigned long) ()

This crash doesn't make a lot of sense. Does it always crash in this place? Can you use the bt command in gdb to see the full stack trace after it crashes?

Also the benchmark uses the same batching technique as test 91, so I'm not sure why one crashes and the other one doesn't.

[thomasvaughan]

[89] Cache initialization: SSSE3              ... PASSED
[90] Cache initialization: AVX2               ... SKIPPED
[91] Hash batch test                          ... 
Program received signal SIGSEGV, Segmentation fault.
0x000000000040d7f0 in randomx::Blake2Generator::checkData(unsigned long) ()
(gdb) bt
#0  0x000000000040d7f0 in randomx::Blake2Generator::checkData(unsigned long) ()
#1  0x000000000040d85e in randomx::Blake2Generator::getUInt32() ()
#2  0x000000000040bab3 in randomx::generateSuperscalar(randomx::SuperscalarProgram&, randomx::Blake2Generator&) ()
#3  0x8d5ac368c67aecb2 in ?? ()
#4  0x00007f7fff269998 in ?? ()
#5  0x6a09e667f2bdc948 in ?? ()
#6  0x0000000004333583 in ?? ()
#7  0x00007ed8f6a272a8 in ?? ()
#8  0x00007ed8f6a1ef40 in ?? ()
#9  0x00007f7fff269960 in ?? ()
#10 0x00007f7fff2699e0 in ?? ()
#11 0x00007ed8f6a1ffd0 in ?? ()
#12 0x0000000000408939 in randomx::initDatasetItem(randomx_cache*, unsigned char*, unsigned long) ()
#13 0x000000000041a1b1 in randomx::InterpretedLightVm<randomx::AlignedAllocator<64ul>, true>::datasetRead(unsigned long, unsigned long (&) [8]) ()
#14 0x000000000041c10c in randomx::InterpretedVm<randomx::AlignedAllocator<64ul>, true>::execute() ()
#15 0x0000000000409e95 in randomx_calculate_hash_next ()
#16 0x0000000000406e06 in void runTest<main::{lambda()#86}>(char const*, bool, main::{lambda()#86}) [clone .constprop.107] ()
#17 0x0000000000415a25 in main ()
(gdb) quit
A debugging session is active.

    Inferior 1 [process 5884] will be killed.

Quit anyway? (y or n) y

[tevador]

@thomasvaughan can you test again using both #187 and #189 (applied at the same time)?

[thomasvaughan]

I refreshed #187 and tried again.

randomx_benchmark is still OK. Here's the last few lines of output from gdb ./randomx-tests:

[89] Cache initialization: SSSE3              ... PASSED
[90] Cache initialization: AVX2               ... SKIPPED
[91] Hash batch test                          ... assertion "equalsHex(hash1, "639183aae1bf4c9a35884cb46b09cad9175f04efd7684e7262a0ac1c2f0b4e3f")" failed: file "/home/t/usr/src/RandomX/src/tests/tests.cpp", line 1072, function "main()::<lambda()>"

Program received signal SIGABRT, Aborted.
0x00007bd92ff678aa in _lwp_kill () from /usr/lib/libc.so.12
(gdb) bt
#0  0x00007bd92ff678aa in _lwp_kill () from /usr/lib/libc.so.12
#1  0x00007bd92ff6715a in abort () at /usr/src/lib/libc/stdlib/abort.c:74
#2  0x00007bd92fea84d7 in __assert13 (file=<optimized out>, 
    line=<optimized out>, function=<optimized out>, failedexpr=<optimized out>)
    at /usr/src/lib/libc/gen/assert.c:72
#3  0x0000000000406eb3 in void runTest<main::{lambda()#86}>(char const*, bool, main::{lambda()#86}) [clone .constprop.107] ()
#4  0x0000000000415a25 in main ()
(gdb) quit
A debugging session is active.

	Inferior 1 [process 4255] will be killed.

Quit anyway? (y or n) y

I also checked out #189, but it didn't compile. I wonder if you've accidentally put the wrong src/virtual_memory.cpp in #189; since I don't see any "defined(__NetBSD__)" in there and, it's falling through to the not-defined-anything case:

[ 26%] Building CXX object CMakeFiles/randomx.dir/src/virtual_memory.cpp.o
/home/t/usr/src/RandomX/src/virtual_memory.cpp: In function ‘void* allocLargePagesMemory(std::size_t)’:
/home/t/usr/src/RandomX/src/virtual_memory.cpp:147:83: error: ‘MAP_HUGETLB’ was not declared in this scope
  mem = mmap(nullptr, bytes, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB | MAP_POPULATE, -1, 0);
                                                                                   ^~~~~~~~~~~

Edit: markup

[thomasvaughan]

I've just realized you said "applied at the same time"! Sorry, will try again.

[thomasvaughan]

Having followed your instructions properly this time, the last few lines of output from ./randomx-tests are now:

[89] Cache initialization: SSSE3              ... PASSED
[90] Cache initialization: AVX2               ... SKIPPED
[91] Hash batch test                          ... PASSED
[92] Preserve rounding mode                   ... PASSED

All tests PASSED
1 tests were SKIPPED due to incompatible configuration (see above)

Also, ./randomx-benchmark is still good.

[tevador]

Cool, so it looks like all issues have been solved.