Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQS output plugin #796

Merged
merged 10 commits into from
Aug 3, 2021
Merged

Conversation

adnan-kobir-sp
Copy link
Contributor

@adnan-kobir-sp adnan-kobir-sp commented Jul 24, 2021

Q A
Bug fix? no
New feature? yes
API breaks? no
Deprecations? no
Related tickets none
License Apache 2.0

What's in this PR?

Adds SQS output plugin

Additional context

To test the implementation:

  • compiled fluentd with the added fluent-plugin-sqs gem and pushed up the docker image to my public repository
  • Modified pkg/sdk/api/v1beta1/logging_types.go and set DefaultFluentdImageRepository to my public image repo temporarily
  • compile the controller and pushed up to my public repository
  • deployed to my local kind cluster
  • followed the guide and deployed the logging demo
  • added the following output
apiVersion: logging.banzaicloud.io/v1beta1
kind: Output
metadata:
  name: sqs-output-sample
  namespace: logging
spec:
  sqs:
    queue_name: some-aws-sqs-queue
    create_queue: false
    region: us-east-1
  • modified the existing Flow and appended sqs-output-sample

I was able to verify that messages were successfully being populated in some-aws-sqs-queue queue

Checklist

  • Code meets the Developer Guide
  • User guide and development docs updated (if needed)
  • Related Helm chart(s) updated (if needed)

@adnan-kobir-sp adnan-kobir-sp changed the title Sqs output plugin SQS output plugin Jul 26, 2021
@ahma ahma requested review from ahma and pepov July 26, 2021 14:53
@ahma
Copy link
Contributor

ahma commented Jul 27, 2021

@adnan-kobir-sp Thanks for your contribution! Could you pls use secret refs. to store aws creds eg.: https://github.com/banzaicloud/logging-operator/blob/master/pkg/sdk/model/output/aws_elasticsearch.go#L86 Thanks!

@adnan-kobir-sp
Copy link
Contributor Author

adnan-kobir-sp commented Jul 28, 2021

@adnan-kobir-sp Thanks for your contribution! Could you pls use secret refs. to store aws creds eg.: https://github.com/banzaicloud/logging-operator/blob/master/pkg/sdk/model/output/aws_elasticsearch.go#L86 Thanks!

@ahma done. Thanks for the review

Copy link
Contributor

@ahma ahma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM Thanks @adnan-kobir-sp !

@adnan-kobir-sp
Copy link
Contributor Author

@pepov can I get a review please? thanks

Copy link
Member

@pepov pepov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@pepov pepov merged commit a14085d into kube-logging:master Aug 3, 2021
@adnan-kobir-sp adnan-kobir-sp deleted the sqs-output-plugin branch August 3, 2021 12:33
bshifter pushed a commit that referenced this pull request Aug 5, 2021
* feat: add aws sqs output
* fix: update v1alpha1 output_types
* feat: add remaining input params
* fix: make generate
* fix: test output order
* fix: use secret refs for aws keys

Co-authored-by: Ferenc HERNADI <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants