Warnings handling

[Korijn]

Let's say that we have the following piece of code under test:

import warnings
import pytest

def foo():
    warnings.warn("bar")

def test_foo():
    with pytest.warns(UserWarning):
        foo()

This test will pass, and no warning will be visible in pytest's output because stdout is suppressed when a test passes.

Now we change the function under test:

def foo():
    warnings.warn("bar")
    warnings.warn("baz", DeprecationWarning)

The test will still pass, and the newly introduced deprecationwarning will be hidden from pytest's output.

I now have to run py.test with the -s argument to make sure I don't miss these things, but this also shows the UserWarning that I am asserting, which is messing up pytest's output!

There are two ways to resolve, in my humble opinion:

• a command line option to cause tests to fail in case of warnings that haven't been caught with pytest.warns
• a command line option to show stdout of tests in case of warnings that haven't been caught with pytest.warns

Is there already a way (or a plugin) to configure pytest as desired? Or does it require new development effort? Would like to hear your thoughts on the matter. Thanks for your time. :)

[RonnyPfannschmidt]

there is the pytest-warnings plugin - it already shows the warnings at the end of a test

the plan is to integrate it into a upcoming py.test feature release

please take a look, we'd appreciate feedback on whether it works for you

[Korijn]

I just tried it, ran py.test with -rw, but it didn't show any warnings in the output... (with the above code)

[The-Compiler]

-rw is about pytest warnings, not python warnings - did you try the pytest-warnings plugin like Ronny suggested?

[Korijn]

I'm pretty sure, yeah! plugins: cov-2.3.1, mock-1.2, warnings-0.1.0, xdist-1.14

https://pypi.python.org/pypi/pytest-warnings

It says to run with -rw...

[The-Compiler]

Oh! Sorry, I didn't know that plugin repurposed the -rw switch 😉

FWIW I use it with -Werror which seems to work fine for me - maybe @fschulze can say more.

[fschulze]

With -rw it should show it in the report, with -Werror it should show it as a test failure.

[Korijn]

Doing some more testing reveals that the plugin doesn't report warnings when combined with pytest-xdist for n>1. It does work when I set n=1.

It does fulfill our needs perfectly!

[RonnyPfannschmidt]

@fschulze any idea how i could debug that?

[fschulze]

I don't know how xdist works. We may be using the wrong hook in pytest-warnings. I have never used xdist myself yet.

[Korijn]

https://github.com/pytest-dev/pytest-xdist/blob/master/xdist/plugin.py

Doesn't look like there is a clash between the hooks used...

[RonnyPfannschmidt]

@fschulze i believe warnings are not transferred between different interpreter processes

[Korijn]

#602

Looks like you're right. Should I go and complain over there? :)

[RonnyPfannschmidt]

superseeded by the xdist issue, thanks for bringing it to attention again