Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GUI tests fail on Ubuntu Focal #493

Closed
apyrgio opened this issue Jul 31, 2023 · 1 comment
Closed

GUI tests fail on Ubuntu Focal #493

apyrgio opened this issue Jul 31, 2023 · 1 comment

Comments

@apyrgio
Copy link
Contributor

apyrgio commented Jul 31, 2023

GUI tests have started to fail on Ubuntu Focal, once we added an unrelated test in our test suite. And by unrelated, we mean that creating even a dummy tests/isolation_provider/test_base.py test that returns immediately makes them error out with:

[...]
tests/gui/test_main_window.py::test_default_menu PASSED                  [ 83%]
tests/gui/test_main_window.py::test_no_update PASSED                     [ 84%]
tests/gui/test_main_window.py::test_update_detected PASSED               [ 85%]
tests/gui/test_main_window.py::test_update_error Fatal Python error: Segmentation fault

Thread 0x00007f49c5d42700 (most recent call first):
  File "/usr/lib/python3.8/traceback.py", line 359 in extract
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/exceptiongroup/_formatting.py", line 96 in __init__
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/exceptiongroup/_formatting.py", line 179 in __init__
  File "/usr/lib/python3.8/traceback.py", line 103 in print_exception
  File "/usr/lib/python3.8/logging/__init__.py", line 626 in formatException
  File "/usr/lib/python3.8/logging/__init__.py", line 676 in format
  File "/usr/lib/python3.8/logging/__init__.py", line 929 in format
  File "/usr/lib/python3.8/logging/__init__.py", line 1085 in emit
  File "/usr/lib/python3.8/logging/__init__.py", line 1187 in emit
  File "/usr/lib/python3.8/logging/__init__.py", line 954 in handle
  File "/usr/lib/python3.8/logging/__init__.py", line 1661 in callHandlers
  File "/usr/lib/python3.8/logging/__init__.py", line 1599 in handle
  File "/usr/lib/python3.8/logging/__init__.py", line 1589 in _log
  File "/usr/lib/python3.8/logging/__init__.py", line 1475 in error
  File "/usr/lib/python3.8/logging/__init__.py", line 1481 in exception
  File "/home/circleci/project/dangerzone/gui/updater.py", line 294 in check_for_updates
  File "/home/circleci/project/dangerzone/gui/updater.py", line 300 in run

Current thread 0x00007f49d6b55740 (most recent call first):
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pytestqt/qt_compat.py", line 162 in exec
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pytestqt/wait_signal.py", line 51 in wait
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pytestqt/wait_signal.py", line 151 in __exit__
  File "/home/circleci/project/tests/gui/test_main_window.py", line 281 in test_update_error
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/python.py", line 194 in pytest_pyfunc_call
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/python.py", line 1788 in runtest
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/runner.py", line 169 in pytest_runtest_call
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/runner.py", line 262 in <lambda>
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/runner.py", line 341 in from_call
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/runner.py", line 261 in call_runtest_hook
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/runner.py", line 222 in call_and_report
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/runner.py", line 133 in runtestprotocol
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/runner.py", line 114 in pytest_runtest_protocol
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/main.py", line 349 in pytest_runtestloop
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/main.py", line 324 in _main
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/main.py", line 270 in wrap_session
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/main.py", line 317 in pytest_cmdline_main
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_callers.py", line 80 in _multicall
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_manager.py", line 112 in _hookexec
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/pluggy/_hooks.py", line 433 in __call__
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/config/__init__.py", line 166 in main
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/lib/python3.8/site-packages/_pytest/config/__init__.py", line 189 in console_main
  File "/home/circleci/.cache/pypoetry/virtualenvs/dangerzone-3aSsmiER-py3.8/bin/pytest", line 8 in <module>
make: *** [Makefile:40: test] Segmentation fault (core dumped)

Exited with code exit status 2

CircleCI received exit code 2

With this small change, and only in our CI runners, CI tests fail reproducibly in Ubuntu Focal environments, and on a specific line. Also, attempting to SSH into our CI runners and run these tests manually does not trigger any error, even in our containerized Dangerzone environments.

So, we don't know the root cause of this error, and we haven't achieved reproducing it in an interactive session, so that we can probe deeper.

@apyrgio
Copy link
Contributor Author

apyrgio commented Jul 31, 2023

While we don't know what's the exact source of the error, given that this only happens in Ubuntu Focal, which still uses Qt5, is probably an indication that there is an issue with shared state between Qt 5, PySide6 and pytest-qt. Also, given that it reproducibly happens in a specific line in the code, it could be that we unwillingly trigger a segfault in that Qt 5 version that has been fixed.

In any case, end users will actually use Dangerzone with PySide2 and pytest-qt will not be involved. To the best of our knowledge, this combination does not trigger any segfaults. For this reason, and in order to avoid future wild goose chases, we should run each GUI test in a separate process.

Turns out that this is not very easy though, due to a PySide issue. To circumvent this, we can change our Makefile to list all the GUI tests and run each one sequentially in a separate pytest invocation. The upside is that this circumvents the PySide issue, but on the flip side, we don't get a summary of all the test runs at the end.

Finally, note that we will use this workaround for all the platforms that we support, even if there are no errors for most of them. For now, this makes sense since we will avoid debugging pytest-qt -related issues for all platforms. We can revisit this decision later on.

@apyrgio apyrgio closed this as completed in a72a319 Aug 1, 2023
almet added a commit that referenced this issue May 23, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
almet added a commit that referenced this issue May 23, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
almet added a commit that referenced this issue May 27, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
almet added a commit that referenced this issue Jun 5, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
almet added a commit that referenced this issue Jun 5, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
almet added a commit that referenced this issue Jun 5, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
almet added a commit that referenced this issue Jun 5, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
almet added a commit that referenced this issue Jun 5, 2024
This is mainly to check if the CI makes it work properly, especially
on Ubuntu Focal, as described in #493
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant