bpo-37179: Support asyncio loop.start_tls() for TLS in TLS #28073
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
|
@asvetlov could you please review this PR from my colleague? 🙏 It also looks like this may help aiohttp gain support for proxying TLS-in-TLS too. |
|
@tiran maybe you could help? |
webknjaz
added a commit
to bmbouter/aiohttp
that referenced
this pull request
Oct 3, 2021
This patch adds full end-to-end tests for sending requests to HTTP and HTTPS endpoints through an HTTPS proxy. The first case is currently supported and the second one is not. This is why the latter test is marked as expected to fail. The support for TLS-in-TLS in the upstream stdlib asyncio is currently disabled but is available in Python 3.9 via monkey-patching which is demonstrated in the added tests. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * aio-libs#5992 Co-Authored-By: Sviatoslav Sydorenko <[email protected]>
webknjaz
added a commit
to bmbouter/aiohttp
that referenced
this pull request
Oct 3, 2021
This patch adds full end-to-end tests for sending requests to HTTP and HTTPS endpoints through an HTTPS proxy. The first case is currently supported and the second one is not. This is why the latter test is marked as expected to fail. The support for TLS-in-TLS in the upstream stdlib asyncio is currently disabled but is available in Python 3.9 via monkey-patching which is demonstrated in the added tests. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * aio-libs#5992 Co-Authored-By: Sviatoslav Sydorenko <[email protected]>
webknjaz
added a commit
to bmbouter/aiohttp
that referenced
this pull request
Oct 3, 2021
This patch adds full end-to-end tests for sending requests to HTTP and HTTPS endpoints through an HTTPS proxy. The first case is currently supported and the second one is not. This is why the latter test is marked as expected to fail. The support for TLS-in-TLS in the upstream stdlib asyncio is currently disabled but is available in Python 3.9 via monkey-patching which is demonstrated in the added tests. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * aio-libs#5992 Co-Authored-By: Sviatoslav Sydorenko <[email protected]>
webknjaz
added a commit
to bmbouter/aiohttp
that referenced
this pull request
Oct 3, 2021
This patch adds full end-to-end tests for sending requests to HTTP and HTTPS endpoints through an HTTPS proxy. The first case is currently supported and the second one is not. This is why the latter test is marked as expected to fail. The support for TLS-in-TLS in the upstream stdlib asyncio is currently disabled but is available in Python 3.9 via monkey-patching which is demonstrated in the added tests. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * aio-libs#5992 Co-Authored-By: Sviatoslav Sydorenko <[email protected]>
webknjaz
pushed a commit
to aio-libs/aiohttp
that referenced
this pull request
Oct 3, 2021
This patch adds full end-to-end tests for sending requests to HTTP and
HTTPS endpoints through an HTTPS proxy. The first case is currently
supported and the second one is not. This is why the latter test is
marked as expected to fail. The support for TLS-in-TLS in the upstream
stdlib asyncio is currently disabled but is available in Python 3.9
via monkey-patching which is demonstrated in the added tests.
Refs:
* https://bugs.python.org/issue37179
* python/cpython#28073
* #5992
Co-authored-by: Sviatoslav Sydorenko <[email protected]>
PR #6002
webknjaz
pushed a commit
to webknjaz/aiohttp
that referenced
this pull request
Oct 3, 2021
This patch adds full end-to-end tests for sending requests to HTTP and HTTPS endpoints through an HTTPS proxy. The first case is currently supported and the second one is not. This is why the latter test is marked as expected to fail. The support for TLS-in-TLS in the upstream stdlib asyncio is currently disabled but is available in Python 3.9 via monkey-patching which is demonstrated in the added tests. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * aio-libs#5992 Co-authored-by: Sviatoslav Sydorenko <[email protected]> PR aio-libs#6002 (cherry picked from commit d66e07c)
webknjaz
added a commit
to aio-libs/aiohttp
that referenced
this pull request
Oct 3, 2021
…nst ``proxy.py`` (#6033) This patch adds full end-to-end tests for sending requests to HTTP and HTTPS endpoints through an HTTPS proxy. The first case is currently supported and the second one is not. This is why the latter test is marked as expected to fail. The support for TLS-in-TLS in the upstream stdlib asyncio is currently disabled but is available in Python 3.9 via monkey-patching which is demonstrated in the added tests. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * #5992 Co-authored-by: bmbouter <[email protected]> Co-authored-by: Sviatoslav Sydorenko <[email protected]> PR #6002 (cherry picked from commit d66e07c)
webknjaz
added a commit
to bmbouter/aiohttp
that referenced
this pull request
Oct 5, 2021
This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * aio-libs#6044 Resolves aio-libs#3816 Resolves aio-libs#4268 Co-Authored-By: Brian Bouterse <[email protected]> Co-Authored-By: Jordan Borean <[email protected]> Co-Authored-By: Sviatoslav Sydorenko <[email protected]>
webknjaz
added a commit
to bmbouter/aiohttp
that referenced
this pull request
Oct 5, 2021
This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * aio-libs#6044 Resolves aio-libs#3816 Resolves aio-libs#4268 Co-Authored-By: Brian Bouterse <[email protected]> Co-Authored-By: Jordan Borean <[email protected]> Co-Authored-By: Sviatoslav Sydorenko <[email protected]>
webknjaz
added a commit
to aio-libs/aiohttp
that referenced
this pull request
Oct 5, 2021
This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * #6044 PR #5992 Resolves #3816 Resolves #4268 Co-authored-by: Brian Bouterse <[email protected]> Co-authored-by: Jordan Borean <[email protected]> Co-authored-by: Sviatoslav Sydorenko <[email protected]>
webknjaz
pushed a commit
to webknjaz/aiohttp
that referenced
this pull request
Oct 5, 2021
This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * aio-libs#6044 PR aio-libs#5992 Resolves aio-libs#3816 Resolves aio-libs#4268 Co-authored-by: Brian Bouterse <[email protected]> Co-authored-by: Jordan Borean <[email protected]> Co-authored-by: Sviatoslav Sydorenko <[email protected]> (cherry picked from commit c29e5fb)
Merged
5 tasks
webknjaz
pushed a commit
to webknjaz/aiohttp
that referenced
this pull request
Oct 5, 2021
This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * aio-libs#6044 PR aio-libs#5992 Resolves aio-libs#3816 Resolves aio-libs#4268 Co-authored-by: Brian Bouterse <[email protected]> Co-authored-by: Jordan Borean <[email protected]> Co-authored-by: Sviatoslav Sydorenko <[email protected]> (cherry picked from commit c29e5fb)
webknjaz
pushed a commit
to webknjaz/aiohttp
that referenced
this pull request
Oct 11, 2021
This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * aio-libs#6044 PR aio-libs#5992 Resolves aio-libs#3816 Resolves aio-libs#4268 Co-authored-by: Brian Bouterse <[email protected]> Co-authored-by: Jordan Borean <[email protected]> Co-authored-by: Sviatoslav Sydorenko <[email protected]> (cherry picked from commit c29e5fb)
webknjaz
pushed a commit
to webknjaz/aiohttp
that referenced
this pull request
Oct 12, 2021
This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * https://bugs.python.org/issue37179 * python/cpython#28073 * https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * aio-libs#6044 PR aio-libs#5992 Resolves aio-libs#3816 Resolves aio-libs#4268 Co-authored-by: Brian Bouterse <[email protected]> Co-authored-by: Jordan Borean <[email protected]> Co-authored-by: Sviatoslav Sydorenko <[email protected]> (cherry picked from commit c29e5fb)
webknjaz
added a commit
to aio-libs/aiohttp
that referenced
this pull request
Oct 12, 2021
…ent (#6049) This patch opens up the code path and adds the implementation that allows end-users to start sending HTTPS requests through HTTPS proxies. The support for TLS-in-TLS (needed for this to work) in the stdlib is kinda available since Python 3.7 but is disabled for `asyncio` with an attribute/flag/toggle. When the upstream CPython enables it finally, aiohttp v3.8+ will be able to work with it out of the box. Currently the tests monkey-patch `asyncio` in order to verify that this works. The users who are willing to do the same, will be able to take advantage of it right now. Eventually (hopefully starting Python 3.11), the need for monkey-patching should be eliminated. Refs: * bugs.python.org/issue37179 * python/cpython#28073 * docs.aiohttp.org/en/stable/client_advanced.html#proxy-support * #6044 PR #5992 Resolves #3816 Resolves #4268 Co-authored-by: Brian Bouterse <[email protected]> Co-authored-by: Jordan Borean <[email protected]> Co-authored-by: Sviatoslav Sydorenko <[email protected]>
jborean93
force-pushed
the
asyncio-ssl-proxy
branch
from
af18b2b to
0aee4f8
Compare
|
I've been trying to solve the test failures on Windows and I think I've found the underlying problem. The current tests have the server sending the shutdown signal on the inner TLS layer which is received by the client. When processed by the client the SSLProtocol is closing the outer protocols and due to how the code is set up the outer TLS layer is being closed before the inner one. More work is most likely needed to solve this problem. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds the
_start_tls_compatibleattribute that start_tls checks. This is required to support using a HTTPS proxy when targeting a HTTPS endpoint.This is my first PR to CPython so I'm unsure what the full process is, let me know if there is anything I'm missing.
https://bugs.python.org/issue37179