Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Std out sink to use event model #599

Merged
merged 7 commits into from
Nov 18, 2021

Conversation

graytaylor0
Copy link
Member

@graytaylor0 graytaylor0 commented Nov 16, 2021

Description

Updated stdout sink to use the Event model

Issues Resolved

Closes #585

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@graytaylor0 graytaylor0 requested a review from a team November 16, 2021 03:11
@codecov-commenter
Copy link

codecov-commenter commented Nov 16, 2021

Codecov Report

Merging #599 (f37ef25) into main (eb94f4a) will decrease coverage by 0.25%.
The diff coverage is 77.77%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #599      +/-   ##
============================================
- Coverage     91.74%   91.49%   -0.26%     
+ Complexity      571      570       -1     
============================================
  Files            72       72              
  Lines          1744     1751       +7     
  Branches        144      145       +1     
============================================
+ Hits           1600     1602       +2     
- Misses          113      114       +1     
- Partials         31       35       +4     
Impacted Files Coverage Δ
...om/amazon/dataprepper/plugins/sink/StdOutSink.java 86.66% <77.77%> (-13.34%) ⬇️
...com/amazon/dataprepper/pipeline/ProcessWorker.java 85.71% <0.00%> (-5.72%) ⬇️
...dataprepper/pipeline/server/DataPrepperServer.java 90.16% <0.00%> (-1.22%) ⬇️
...ataprepper/model/buffer/SizeOverflowException.java 100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eb94f4a...f37ef25. Read the comment docs.

@graytaylor0 graytaylor0 changed the title Std out sink to event Std out sink to use event model Nov 16, 2021
private final Record<String> TEST_RECORD_1 = new Record<>(TEST_DATA_1);
private final Record<String> TEST_RECORD_2 = new Record<>(TEST_DATA_2);
private final List<Record<String>> TEST_RECORDS = Arrays.asList(TEST_RECORD_1, TEST_RECORD_2);
private final Map<String, Object> TEST_DATA_1 = new HashMap<>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend that you make this a non-final and initialize it in the setup. As it is right now, authors will need to clear the map in-between tests.

The same goes for TEST_DATA_2 and TEST_RECORDS.

Signed-off-by: Taylor Gray <[email protected]>
@@ -32,7 +32,7 @@ jacocoTestCoverageVerification {
violationRules {
rule { //in addition to core projects rule
limit {
minimum = 0.90
minimum = 0.89
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep the threshold as is. There is nothing in this PR that requires us to lower it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would think so, but the coverage is now .89 after this PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran your changes locally addressing the Junit issue and looking at the code coverage results from the build. It is possible to achieve .90. Let's keep it at .90.

Copy link
Member Author

@graytaylor0 graytaylor0 Nov 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had tried this yesterday and it did fix it. Had bumped it back to 0.90 for the next revision. Interesting little thing there with the 2 different versions of Junit. Thanks for catching it!

Comment on lines 19 to 20
import org.junit.Test;
import org.junit.jupiter.api.BeforeEach;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are using two different version of Junit now. I believe this is what is impacting your test coverage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! It's a subtle difference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making it support both! I have one comment on variable names and then I'll be ready to approve. It looks like the 90% coverage is working again, correct?

Yep .90 is working again

Copy link
Member

@dlvenable dlvenable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making it support both! I have one comment on variable names and then I'll be ready to approve. It looks like the 90% coverage is working again, correct?

private final Record<String> TEST_RECORD_1 = new Record<>(TEST_DATA_1);
private final Record<String> TEST_RECORD_2 = new Record<>(TEST_DATA_2);
private final List<Record<String>> TEST_RECORDS = Arrays.asList(TEST_RECORD_1, TEST_RECORD_2);
List<Record<Object>> TEST_RECORDS;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field and the variables in setup should be renamed to standard Java field and variable conventions.

Signed-off-by: Taylor Gray <[email protected]>
Signed-off-by: Taylor Gray <[email protected]>
@graytaylor0 graytaylor0 merged commit 8dd106b into opensearch-project:main Nov 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

StdOut Sink to use Event Model
4 participants