Inconsistent results with OpenMP resulting in HT3D bugs

[Vojta-Salek]

Describe the bug
I was testing whether the HT3D bugs (#13361) related to the order of MATL lines in the input code were fixed for more complex geometries. I noticed different results when running FDS while specifying OpenMP. The bugs appear to be fixed when running fds CHID.fds. However, when using fds_local -o 3 CHID.fds, I still observe differences in the outputs due to HT3D bugs. I am creating this issue in case the problem is not related to an incorrect installation or execution on my end.

My approach
I wanted to ensure fds and fds_local commands function identically. I noticed different compilation dates, but everything else appears the same for the following CMD inputs:

• fds -> Screenshot A
• fds CHID.fds -> Screenshot A
• fds_local -v -> Screenshot A
• fds_local -o 3 CHID.fds -> Screenshot B

The issue persists whether using the special CMDfds command prompt or a standard CMD window. At a minimum, it seems odd that fds_local -v reports a different compilation time compared to fds_local -o 3 CHID.fds. Maybe the problem is somewhere else, this is the only difference I found.

Could you please help me run the simulation with increased OpenMP while avoiding the already fixed bugs?

Screenshots
Screenshot A

Screenshot B

Versions:

• OS: Win10 Pro 22H2
• FDS: FDS-6.9.1-1751-gdf6cc22-nightly

Thank you as always!

[mcgratta]

Thanks, I'll take a look.

[mcgratta]

The difference compilation times is easy to explain --- these are two different executables. When OpenMP directives are included in the compilation, the Intel Fortran compiler changes its optimization strategy for parallelized loops. If a loop is executed with multiple threads, which is how OpenMP works, then the compiler cannot use other optimization strategies to make the computation go faster. The source code for the two executables is exactly the same, but the compilation is different.

Is there a particular case you have where the results are different when run with OpenMP. With multiple OpenMP threads, the results should be exactly the same. By results, I mean the numbers that are computed. We have some test cases to check that the OpenMP and non-OpenMP executables produce the same results, but there is always a possibility that a bug causes the multiple threads to produce a different result.

[Vojta-Salek]

Thank you for the explanation on the compilation times. I see differences for instance for the cases provided in #13361 in this reply when using fds_local -o 3 CHID.fds. When using fds CHID.fds, all results are identical.

[mcgratta]

OK, this is another good bug. The problem is that FDS divides the processing of boundary cells among the OpenMP threads. Two threads are "touching" the same value on either side of the obstruction. This will require some minor surgery to fix. Until then, I would avoid using the OpenMP with HT3D.

[Vojta-Salek]

That would also explain why my simulations reported some differences while yours did not in the previous bug report. Understood, thank you for attempting the surgery.