Linux/macOS CI remove excess logging, improve remaining output + other tidyup

[rhuanjl]

Improve runtests.py

• remove unnecessary logfile
• print a welcome message saying that the tests are being run and how many threads are being used
• don't print individual test passes by default (add show-passes flag for printing these if wanted)
• don't print "Running interpreted variant" twice
• print results for a set of tests including time taken before beginning the next set
• other formatting improvements
• remove defunct logic and switches related to ChakraFull

[rhuanjl]

The macOS debug build wasn't any faster so maybe this change isn't worth it - though will be good to see how long the other ones take.

[rhuanjl]

Whilst the macOS debug build was unchanged. the other builds did seem to be a little faster - 50/50 if this should be merged.

[ppenzin]

I am hugely in favor of reducing amount of output - even if it does not look that bad in terminal (because we are rewriting lines), we are actually printing a line for every execution of a test.

Looks like --logfile replicates all output into a file, we might need a log, but not that detailed by default :)

The part of the change reducing the output looks good, but I am not sure about process count, default looks reasonable for user machine (number of processors). We can probably add a CMake variable to override the default and set it for CI only.

[rhuanjl]

Significant update:

1. I reverted the process change (briefly considered doing cpu_count()-1 but left as just cpu_count() for now)
2. Removed the logfile completely (omitting the parameter before merely resulted in using a default path) -if we want to log the output in a file when running offline can do this by capturing the terminal output
3. Improved the terminal output, a welcome message is printed stating the build type being tested and the number of threads being used (matching the windows output) and the description of the test type (interpreted, dynopogo or disable_jit) being run is displayed before testing commences, and the results of that type are shown as soon it completes before beginning the next type, also a time taken is shown for the type.
4. I've edited the verbose flag so it also enables the show-passes flag, so verbose on its own will give the most detailed level of output.
5. Whilst doing 3 I noticed that the reason it used to say "Running Interpreted Variant" twice was actually a logical error in when the print() commands were called - it wasn't running any extra tests - I've fixed the error so it no longer says this twice.
6. Drive by cleanup: removed various traces of testing for ChakraFull/chakra.dll which used to run against the same test suite, this included removing 'exclude_jshost' a flag that was used to mark tests meant for ChakraCore only to avoid running them when testing ChakraFull and 'exclude_ch' a defunct flag that used to be used to mark tests meant for ChakraFull only (all such tests had already been updated to support ChakraCore or deleted).

[rhuanjl]

One note on performance - having added the welcome message that prints the process_count can see that the macOS machines have 3 cores and the Ubuntu machines only 2. I think the windows machines have 4.

[rhuanjl]

As general tidy-up and improvement of the console output I think this is good to merge, any thoughts? @ppenzin

I've noticed another couple of defunct flags exclude_chk and exclude_fre to remove (chk is an old alias for Debug and fre is an old alias for Test - nothing uses these flags though) but I'll put that in a future PR when I have something else to submit.

[ppenzin]

I think this is good, we can wait for the final job and then merge. It would be good to add some form of progress indicator to see if a test is hanging, but it is not urgent.

[rhuanjl]

Thanks for the review.

I think this is good, we can wait for the final job and then merge. It would be good to add some form of progress indicator to see if a test is hanging, but it is not urgent.

There's already a timeout (with an automatic fail) if an individual test takes more than a minute, I thought that was sufficient - whilst it would be nice to know the progress aside from specific hangs it's roughly the same each time so didn't think it was needed, offline can use the --show-passes flag to see it if you need it.