Resurrect reporting to codecov

[webknjaz]

I noticed that during the GHA migration this integration has been lost (cde5f20#diff-6ac3f79fc25d95cd1e3d51da53a4b21b939437392578a35ae8cd6d5366ca5485L74). It should probably be reintroduced.

[agronholm]

Yeah, it didn't work properly with GHA but it has been resolved in coveralls since.

[webknjaz]

Well, their official action supports tokenless runs for quite a while now. So it should be fine to use it. Or do you intend to migrate to coveralls?

[agronholm]

Ah, sorry, I use coveralls in my own projects so I mixed them up. I'll try to restore this to the workflow and see what happens. The thing is, I'm not sure how to implement parallel coverage reporting with codecov, so if you have any pointers, those would be welcome.

[webknjaz]

You don't need to do anything extra. Just add a reporter to your jobs. Something like https://github.com/ansible/pylibssh/blob/1af102d/.github/workflows/build-test-n-publish.yml#L787-L790 should be enough — codecov merges multiple reports on their side.

Oh, and you probably want to add something like --cov-report=xml:pytest-cov.xml at https://github.com/pypa/wheel/blob/41b375e/setup.cfg#L55 (example: https://github.com/ansible/pylibssh/blob/1af102d/pytest.ini#L26) — I don't see the XML report emitted anywhere.

[agronholm]

I'm seeing codecov falsely reporting coverage readings before the test suite has even finished. How could it have worked without being told that more results are coming in after all?

[webknjaz]

There's some timeout. I think it waits for the PR checks too. But if something takes too long and the jobs report later, codecov updates the coverage.

[agronholm]

Seems like you're correct – it just confused me that codecov reported a lower coverage value before all the jobs had finished. Sorry for the noise.