Automated cross-version testing #159
Conversation
move test of existing files to test class
|
@rocky responding to your comment in the other PR I made;
I will need to work on deciding if its best to "massage" an xdis bytecode to be more like the dis bytecode, or just loosen tests more (I already have excluded argrepr from these tests for example). |
|
@2elli Good idea and I am glad to see this done. When I will get a chance I will try this out!
Farily recently, But beware, because |
|
Perfect, thanks for the reminders! I'll see if classic mode will work better for this. The different versions changing output for dis is important to consider so thank you for bringing that up. One of the next steps I think would be best is to make some tests version specific. For example I would like to include 'co_lnotab' as part of the tests but that is specific for some versions of dis, even if present in xdis. Lastly, let me know if running the tests with pyenv-virtualenv gives you any issues. I wanted tox to be able to use pyenv-virtualenv to discover environments so I used https://github.com/un-def/tox-pyenv-redux to handle that, but I believe there may be a better solution. Thanks :) |
|
I tried running this morning after pip installing What I am supposed to see? What is going wrong? |
|
@rocky It seems everything ran correctly here. I think my wording and the way these tests are setup may be a bit confusing, sorry. I may refactor to something more clear like "prepare_tests". The The readability of tests right now is definitely hurting due to |
|
Ok. Thanks for the information. |
|
@rocky I think these tests should be good for development use now. With these tests, I already have been finding some inaccuracies. Most are inconsequential differences between argvals, but I found incorrect line numbers in 3.10 again. @jdw170000 found that the python provided in https://github.com/python/cpython/blob/3.10/Objects/lnotab_notes.txt is actually incorrect when compared with the C. @jdw170000 and I worked on a fix for xdis that I will open a new PR for. |
…into cross_version_tests
Co-authored-by: jdw170000 <[email protected]>
Many thanks. I will look at this when I get a chance.
Ok. Good news. Keep up the good work! |
… to define which parts of bytecode should be serialized
I wrote this automated cross version tests for checking xdis bytecode accuracy against a dis bytecode using
pyenv,pyenv-virtualenv,tox, andpytest. I this this will be very useful for testing accuracy against permutations of all the modern python versions being used for xdis so we can catch some of the cases that we ran into issues with last year, for example, the line table inaccuracies in 3.10 when running xdis from a different version.There is more work needed to have this be useful but I wanted to create this PR for discussion.
Test Overview
To test cross version accuracy of xdis against dis, I wrote a simple way to "serialize" a bytecode into text. This serialization works the same on an xdis and dis bytecode.
tox_prepare.ini, for each test env,./templates/source/*.pyto./templates/compiled/<version>/*.pyc./templates/serialized/<version>/*.txt.tox.inifor each test env, for each compiled version,./templates/compiled/<version>/<file>.pycand its dis serialized txt file./templates/serialized/<version>/<file>.txtTO-DO
tox-pyenv-reduxfor venv discovery.