Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove AggregateExpr trait #12096

Merged
merged 17 commits into from
Aug 27, 2024
Merged

Remove AggregateExpr trait #12096

merged 17 commits into from
Aug 27, 2024

Conversation

lewiszlw
Copy link
Member

Which issue does this PR close?

Closes #11810.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added physical-expr Physical Expressions optimizer Optimizer rules core Core DataFusion crate proto Related to proto crate labels Aug 21, 2024
@alamb alamb marked this pull request as draft August 21, 2024 17:09
@alamb
Copy link
Contributor

alamb commented Aug 21, 2024

Marking as draft as CI is currnetly not passing so I don't think this is ready for review

@lewiszlw lewiszlw marked this pull request as ready for review August 22, 2024 03:29
@@ -229,7 +230,7 @@ mod tests {
fn partial_aggregate_exec(
input: Arc<dyn ExecutionPlan>,
group_by: PhysicalGroupBy,
aggr_expr: Vec<Arc<dyn AggregateExpr>>,
aggr_expr: Vec<Arc<AggregateFunctionExpr>>,
Copy link
Contributor

@jayzhan211 jayzhan211 Aug 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If AggregateFunctionExpr is not cloned massively, we could remove Arc, similar to that we doesn't wrap Arc for Expr.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could try this in a followup pr.

Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lewiszlw for the clean-up. To avoid breaking any downstream, let's keep this PR focused solely on the removal of the AggregateExpr trait by moving all its methods to AggregateFunctionExpr. Then in a new PR, we can focus on determining what we really need and what can be removed in DF.

@ozankabak
Copy link
Contributor

Thanks for working on this @lewiszlw . As @berkaysynnada mentioned, let's keep all the methods please.

BTW, I don't see a way to remove with_new_expressions without breaking downstream. We also plan to contribute some code upstream in the near future that uses this functionality, so it is probably here for good. However, it may be possible to remove all_expressions (the functionality may be implemented with existing methods - though I should verify that).

We can explore if that is possible and give you an indication next week so we can plan the follow on work.

Thanks again for the awesome collaboration 🚀

@lewiszlw
Copy link
Member Author

lewiszlw commented Aug 22, 2024

Now I think there is no additional change breaking downstream.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lewiszlw @jayzhan211 and @ozankabak

I skimmed this PR and it looks like an improvement to me. Given there are already a good number of eyes on this PR, I'll defer review to other community members. Let me know if I can help

Copy link
Contributor

@jayzhan211 jayzhan211 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@ozankabak
Copy link
Contributor

Let's have @berkaysynnada take one final look tomorrow and then merge 🚀

@berkaysynnada
Copy link
Contributor

I did a final review and I think everything looks good. A nice simplification step, thank you @lewiszlw.

@alamb alamb merged commit 2ac0842 into apache:main Aug 27, 2024
26 checks passed
@alamb
Copy link
Contributor

alamb commented Aug 27, 2024

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate optimizer Optimizer rules physical-expr Physical Expressions proto Related to proto crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove AggregateExpr abstraction layer
5 participants