[CEL] Common Expression Language Custom Task

[jerop]

Changes

This is the initial of an experimental project that provides support for Common Expression Language (CEL) in Tekton Pipelines.

The functionality is provided by a controller that implements the Custom Task interface. Its use cases include evaluating complex expressions to be used in WhenExpressions in subsequent Tasks to guard their execution.

To evaluate a CEL expressions using Custom Tasks, we need to define a Run type with apiVersion: cel.tekton.dev/v1alpha1 and kind: CEL. The Run takes the CEL expressions to be evaluated as Parameters.

As the Run executes, its status field accumulates information about the execution status of the Run in general. If the evaluation is successful, it will also contain the Results of the evaluation under status.results with the corresponding names of the CEL expressions as provided in the Parameters. If no CEL expressions are provided, any CEL expression is invalid or there's any other error, the CEL Custom Task will fail and the details will be included in status.conditions.

Related issues:

• Proposal: Common Expression Language Custom Task (CELRun) community#304
• Common Expression Language Custom Task pipeline#3149
• Feature: expression support in $()  pipeline#2812
• Feature: method to absorb git-source yaml into Pipeline/Task $inputs etc pipeline#2219
• Simple if/conditional support for skipping pipeline#2127
• Consider removing type from params (or _really_ support types) pipeline#1393

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

• Includes docs (if user facing)
• Commit messages follow commit message best practices
• Commit messages includes a project tag in the subject - e.g. [OCI], [hub], [results], [task-loops]

See the contribution guide for more details.

[imjasonh]

This looks great! Just a couple comments about docs.

[imjasonh]

I think we want to pull out the param named expression, fail if it isn't found, and only evaluate that param value. This should also fail if any other params are defined, since that's probably a typo/bug.

I don't think we want/need to support multiple expressions in one run, since it's a bit unclear how it should work, and it's probably clearer (though much more verbose) to just have multiple CEL-typed Runs if you want that.

[jerop]

since it's a bit unclear how it should work, and it's probably clearer (though much more verbose) to just have multiple CEL-typed Runs if you want that.

the way I have it implemented now is that the Run will evaluate each Parameter and produce a Result with the corresponding name (i.e. parameter name == result name), did that to reduce the verbosity and evaluate multiple expressions in one pod when appropriate, what do you think of that?

[imjasonh]

Yeah, I think I'm coming around to it. It is better than having N single-expression Runs.

One possible confusing thing is that if a CEL Run has three expression params and one is an invalid expression, will it fail but still produce results for the other two expressions? Will it still evaluate params defined after the invalid expression? There are some options:

1. Fail and don't report any results if any expression param is invalid.
2. (this is what this PR does) Evaluate each expression in the order they're defined in params, fail when one is invalid and only report results for previous valid evaluations.
3. Evaluate each expression, fail when one is invalid but keep evaluating params and report results for any that are valid expressions. You can report either the first, or all, invalid expressions in status.

Either way I don't think it should block this PR, it'd be better to get usage and feedback.

[souleb]

As a user I would expect it to fail if any param expression is invalid. I think I assume a full validation before it runs

[jerop]

I've added an earlier validation to check that that at least one expression is provided and that the expression types are valid

To be consistent with Tasks which don't produce Results when they fail, I've modified so that we go with option 1 where we don't report any evaluation result if any expression is invalid or has any syntax/evaluation error -- and I agree that we can revisit those options based on user feedback

[imjasonh]

/lgtm

[imjasonh]

/lgtm

[dibyom]

/cc @bigkevmcd

[tekton-robot]

@dibyom: GitHub didn't allow me to request PR reviews from the following users: bigkevmcd.

Note that only tektoncd members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @bigkevmcd

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bigkevmcd]

Looks really good to me, thanks for taking this on.

[imjasonh]

/lgtm

[bigkevmcd]

/approve

[imjasonh]

/approve

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bigkevmcd, ImJasonH

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ImJasonH]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[pritidesai]

haha, I was still reviewing it 🙃 very excited for this 😍

[jerop]

haha, I was still reviewing it 🙃 very excited for this 😍

this is experimental so welcoming your feedback and we can modify as needed, could be easier to discuss in the TEP: tektoncd/community#314

[bobcatfish]

hey @jerop sorry for the late comment, just wondering' what's the type and structure of the result? (e.g. json? or some CEL specific thing? maybe we can link to some CEL docs about this?)