feat: date_add and date_sub functions

[mbutrovich]

Which issue does this PR close?

Does not close an individual PR, but addresses an item in #858.

Rationale for this change

What changes are included in this PR?

New DateAdd and DateSub functions in QueryPlanSerde. They have two arguments: a date and an int32 representing the number of days. DataFusion does not fully support date_add, so this PR introduces functions using the scalar UDF interface. These functions use DataFusion interfaces to apply Arrow kernels to perform the computation.

I also had to generate new q72 reference plans because it uses date_add.

How are these changes tested?

New test cases for date_add and date_sub, both scalar and array arguments. fix: ReusedExchangeExec + CometShuffleExchangeExec under QueryStageExec + should be CometRoot in CometExecSuite also exercises date_add. q72 also exercises date_add as well.

[mbutrovich]

I'll drive into this test failure tomorrow. My assumption of the second expression for date_add always being a Literal does not appear sound.

Cause: org.apache.comet.CometNativeException: General execution error with reason: Not implemented: DateAdd { left: Some(Expr { expr_struct: Some(Literal(Literal { datatype: Some(DataType { type_id: Date, type_info: None }), is_null: false, value: Some(IntVal(0)) })) }), right: Some(Expr { expr_struct: Some(Cast(Cast { child: Some(Expr { expr_struct: Some(Remainder(Remainder { left: Some(Expr { expr_struct: Some(Bound(BoundReference { index: 0, datatype: Some(DataType { type_id: Int64, type_info: None }) })) }), right: Some(Expr { expr_struct: Some(If(IfExpr { if_expr: Some(Expr { expr_struct: Some(Eq(Equal { left: Some(Expr { expr_struct: Some(Literal(Literal { datatype: Some(DataType { type_id: Int64, type_info: None }), is_null: false, value: Some(LongVal(4)) })) }), right: Some(Expr { expr_struct: Some(Literal(Literal { datatype: Some(DataType { type_id: Int64, type_info: None }), is_null: false, value: Some(LongVal(0)) })) }) })) }), true_expr: Some(Expr { expr_struct: Some(Literal(Literal { datatype: Some(DataType { type_id: Int64, type_info: None }), is_null: true, value: None })) }), false_expr: Some(Expr { expr_struct: Some(Literal(Literal { datatype: Some(DataType { type_id: Int64, type_info: None }), is_null: false, value: Some(LongVal(4)) })) }) })) }), fail_on_error: false, return_type: Some(DataType { type_id: Int64, type_info: None }) })) }), datatype: Some(DataType { type_id: Int32, type_info: None }), timezone: "America/Los_Angeles", eval_mode: Legacy, allow_incompat: false })) }) }.

[mbutrovich]

q72 comparison (only SF 1 locally), last two rows of each (using Comet Exec) are relevant. main branch:

TPCDS Snappy:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q72                                                7047           7848        1133          2.2         459.1       1.0X
q72: Comet (Scan)                                  6824           7291         660          2.2         444.6       1.0X
q72: Comet (Scan, Exec)                           10816          10896         112          1.4         704.7       0.7X
q72: Comet (Exec)                                 10811          10959         209          1.4         704.4       0.7X

This PR:

TPCDS Snappy:                             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q72                                                6848           6992         204          2.2         446.2       1.0X
q72: Comet (Scan)                                  6814           6819           7          2.3         444.0       1.0X
q72: Comet (Scan, Exec)                           10376          10442          93          1.5         676.1       0.7X
q72: Comet (Exec)                                 10472          10513          58          1.5         682.3       0.7X

[andygrove]

LGTM with one nit. Thanks @mbutrovich!

[mbutrovich]

I added support for Int8 and Int16. I don't love this control flow. @viirya suggested macros, but I'm open to other ways to approach this.

[viirya]

nit: style

Suggested change

	}
	macro_rules! array_date_arithmetic {
	}
	macro_rules! array_date_arithmetic {

[viirya]

Thanks @mbutrovich @andygrove @kazuyukitanimura