Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added SASL/OAuthBearer #630

Merged
merged 12 commits into from
Jun 17, 2020
Merged

Added SASL/OAuthBearer #630

merged 12 commits into from
Jun 17, 2020

Conversation

oulydna
Copy link
Contributor

@oulydna oulydna commented Jun 10, 2020

Changes

Fixes issue #618

Added OAUTHBEARER as a new sasl_mechanism. The implementation is mainly based on this PR in kafka-python.

Checklist

  • I think the code is well written
  • Documentation reflects the changes
  • Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<type> (e.g. 588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the PR
    • ensure type is one of the following:
      • .feature: Signifying a new feature.
      • .bugfix: Signifying a bug fix.
      • .doc: Signifying a documentation improvement.
      • .removal: Signifying a deprecation or removal of public API.
      • .misc: A ticket has been closed, but it is not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: Fix issue with non-ascii contents in doctest text files.

@codecov
Copy link

codecov bot commented Jun 10, 2020

Codecov Report

Merging #630 into master will decrease coverage by 0.49%.
The diff coverage is 41.46%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #630      +/-   ##
==========================================
- Coverage   98.33%   97.84%   -0.50%     
==========================================
  Files          31       31              
  Lines        5118     5159      +41     
==========================================
+ Hits         5033     5048      +15     
- Misses         85      111      +26     
Impacted Files Coverage Δ
aiokafka/consumer/consumer.py 98.51% <ø> (ø)
aiokafka/producer/producer.py 99.50% <ø> (ø)
aiokafka/conn.py 94.05% <34.37%> (-5.21%) ⬇️
aiokafka/abc.py 68.75% <62.50%> (-6.25%) ⬇️
aiokafka/client.py 97.79% <100.00%> (+<0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68be906...3c03f6c. Read the comment docs.

@lgtm-com
Copy link

lgtm-com bot commented Jun 10, 2020

This pull request introduces 1 alert when merging 4d177db into 10bdd29 - view on LGTM.com

new alerts:

  • 1 for Unused import

aiokafka/conn.py Outdated

def _build_oauth_client_request(self):
return "n,,\x01auth=Bearer {}{}\x01\x01".format(
self._sasl_oauth_token_provider.token(), self._token_extensions()
Copy link
Member

@tvoinarovskyi tvoinarovskyi Jun 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to be able to have token as an async def function. The library is asynchronous, so would assume many people would prefer it that way.

Something like:

def __init__(self):
    self._token_sent = False

async def step(self, payload):
    assert not self._token_sent, "Server should either close or send an empty response"
    token = await self._sasl_oauth_token_provider.token()
    token_extensions = self._token_extensions()
    self._token_sent = True
    return self._build_oauth_client_request(token, token_extensions)

What do you think? If people need to provide synchronous callbacks they can do so by calling:

async def token(self):
    return asyncio.get_running_loop().run_in_executor(None, self._token)
def _token(self):
    # The actual token function implementation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we return None based on self._token_sent?

sasl_plain_username (str): username for sasl PLAIN authentication.
Default: None
sasl_plain_password (str): password for sasl PLAIN authentication.
Default: None
sasl_oauth_token_provider (kafka.oauth.abstract.AbstractTokenProvider):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we were to go with an asynchronous implementation we would need our own abstract class. You can put it in aiokafka.abc.

@tvoinarovskyi
Copy link
Member

@oulydna Great thanks for working on this! I think the code is good to merge, but if you have time to invest into changing the implementation to an asynchronous callback I would be really grateful.
Will need some tests to be added too, but that involves docker container changes, so I can do it after. Did you have the time to test it manually on a real Kafka broker?

@oulydna
Copy link
Contributor Author

oulydna commented Jun 10, 2020

Thanks @tvoinarovskyi. Really appreciate the quick reply.

Yes, I tested it in our own environment.

Good point on the async callback. I will look into this.

@oulydna
Copy link
Contributor Author

oulydna commented Jun 15, 2020

@tvoinarovskyi can you please have a look at the new changes?

@tvoinarovskyi
Copy link
Member

@oulydna Looks good for a merge, few minor points if you have time and let's ship it 🚢

oulydna and others added 5 commits June 17, 2020 18:29
* hotfix/ memory leak in comsumer getmany

* hotfix/ Added changes in change log

* hotfix/ Fixed case where pending future set is empty

* hotfix/ renamed pending future variable
@tvoinarovskyi
Copy link
Member

Thanks again!

@tvoinarovskyi tvoinarovskyi merged commit ea1aab8 into aio-libs:master Jun 17, 2020
@oulydna
Copy link
Contributor Author

oulydna commented Jun 17, 2020

Thanks you @tvoinarovskyi . Really glad to have this integrated. We like aiokafka, and will like it better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants