ROBUSTNESS: Automatically disable multi-threading in forked processing ("multicore")

[HenrikBengtsson]

Forked parallel processing and multi-threading are not the best friends, e.g. futureverse/future.apply#29 (comment). I think it might be worth having future disable multi-threading in such cases by default.

One solution would be do depend on RhpcBLASctl and use:

RhpcBLASctl::blas_set_num_threads(1L)
RhpcBLASctl::omp_set_num_threads(1L)

See also

• Problem with multicore on macOS future.apply#29 (comment) (Problem with multicore on macOS)
• XGBoost parallelization fails when twoClassSummary is the first summaryFunc used topepo/caret#1106 (comment) (XGBoost parallelization fails when twoClassSummary is the first summaryFunc used)
• Issue nested futures that doesn't terminate #348 (nested futures that doesn't terminate)
• Issue detect forking #224 (detect forking)

[EmilRehnberg]

Thanks for all the digging @HenrikBengtsson I wouldn't have been able to diagnose this myself well enough.

I think the solution makes sense AFAIU it anyways. Still having 7 threads running while forking (as you mentioned in #348 ) doesn't seem helpful.

[HenrikBengtsson]

Next release will have an optional prototype/beta feature for temporarily disabling multi-threading in futures using OpenMP. I'm holding back on ditto BLAS for the reason that I don't know how to get the current number of BLAS threads, i.e. I don't know what the corresponding "getter" for RhpcBLASctl::blas_set_num_threads(1L) is.