Segfault in 1.1.3 test suite

[jagerman]

When I build 1.1.3 (on amd64, debian sid, on a Ryzen cpu) and run ./randomx-tests I get a segfault (or sometimes Illegal instruction or some other failure) here which didn't happen in 1.1.2.

[79] Hash test 1a (interpreter)               ... PASSED
[80] Hash test 1b (interpreter)               ... PASSED
[81] Hash test 1c (interpreter)               ... PASSED
[82] Hash test 1d (interpreter)               ... PASSED
[83] Hash test 1e (interpreter)               ... PASSED
Segmentation fault

I bisected it to the PR #133 merge commit. I've tried a few different gcc and clang versions and all fail at the same place. Edit: also tried on an intel box, same failure.

[jagerman]

But I don't see the segfault on the #134 PR branch.

[hyc]

ASAN says there's a heap use-after-free bug here. Looking into it.

[tevador]

@jagerman I fixed one bug in the tests in #134 here: https://github.com/tevador/RandomX/pull/134/files#diff-dcda6435c83281150294cbf602037aeeR1001-R1002

Looks like it's exactly where you get your segfault.

[hyc]

The bug is in tests.cpp. At line 1001 https://github.com/tevador/RandomX/blob/master/src/tests/tests.cpp#L1001 it destroys the vm, but doesn't set the vm pointer to NULL. Then it calls initCache(), which tries to use the vm pointer because it's non-NULL. https://github.com/tevador/RandomX/blob/master/src/tests/tests.cpp#L26

[tevador]

Fixed in 1.1.4