Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update for Pulsar 2.9 #300

Closed
EronWright opened this issue Oct 16, 2021 · 2 comments
Closed

Update for Pulsar 2.9 #300

EronWright opened this issue Oct 16, 2021 · 2 comments
Assignees
@gaoran10
Labels
type/feature Indicates new functionality

Comments

@EronWright
Copy link
Contributor

EronWright commented Oct 16, 2021 •
edited
Loading

Please update the codebase for Pulsar 2.9. I observe there are dependencies on PulsarService::getLocalZkCache (which was removed in 2.9).

The topic lookup aspect seems in need of improvement. Be aware that most ZK structures use the broker's canonical name as the key, which is based on that broker's advertisedAddress:webServicePort. This is not the same as the broker service URL that is returned from the LookupService. The lookup service returns information from advertisedListeners which isn't necessarily the same value.

Related:
@EronWright EronWright added the type/feature Indicates new functionality label Oct 16, 2021
@EronWright EronWright changed the title Update for Pulsar 2.9 (ZooKeeper) Update for Pulsar 2.9 Oct 16, 2021
@gaoran10
Copy link
Collaborator

Thanks for your suggestion.

The method PulsarService::getLocalZkCache was called in method AmqpCommandDecoder::getProtocolDataToAdvertise, it seems that this method isn't used, I'll remove it.

@gaoran10 gaoran10 self-assigned this Oct 17, 2021
@EronWright
Copy link
Contributor Author

EronWright commented Oct 17, 2021 •
edited
Loading

@gaoran10 here's a related PR with a possible solution to the other compilation issue: #299

@EronWright EronWright mentioned this issue Oct 18, 2021
Merged
1 task
gaoran10 added a commit that referenced this issue Oct 26, 2021
@EronWright @gaoran10
[bugfix] Support for broker advertised listeners (#299)
1e5af44 
Master Issue: #300 

### Motivation

This PR reworks the broker service lookup logic for compatibility with 2.9.  The compat issues are:
1. removal of `PulserService::getLocalZkCache` and increased encapsulation of ZooKeeper.  
2. improve compatibility with `advertisedListeners` (don't compare broker service url with web service url).

### Modifications

- Update Pulsar dependency to `2.9.0-rc-202110152205`
- Update `PulsarServiceLookupHandler` to use `NamespaceService` and `MetadataStoreCacheLoader`.
- No need for `PulsarClient` in the lookup handler
- Set `amqp+ssl://` as the scheme that would be used for (as-yet unimplemented) TLS support
- New config option `brokerLookupTimeoutSeconds`

Co-authored-by: gaoran10 <[email protected]>
gaoran10 added a commit that referenced this issue Oct 26, 2021
@EronWright @gaoran10
[bugfix] Support for broker advertised listeners (#299)
c193f6a 
Master Issue: #300 

### Motivation

This PR reworks the broker service lookup logic for compatibility with 2.9.  The compat issues are:
1. removal of `PulserService::getLocalZkCache` and increased encapsulation of ZooKeeper.  
2. improve compatibility with `advertisedListeners` (don't compare broker service url with web service url).

### Modifications

- Update Pulsar dependency to `2.9.0-rc-202110152205`
- Update `PulsarServiceLookupHandler` to use `NamespaceService` and `MetadataStoreCacheLoader`.
- No need for `PulsarClient` in the lookup handler
- Set `amqp+ssl://` as the scheme that would be used for (as-yet unimplemented) TLS support
- New config option `brokerLookupTimeoutSeconds`

Co-authored-by: gaoran10 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gaoran10 gaoran10

Labels
type/feature Indicates new functionality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@EronWright @codelipenghui @gaoran10