Add conditional to DropEventsProcessor

[sbayer55]

Is your feature request related to a problem? Please describe.
Data Prepper currently does not provide any control logic within pipelines.

Describe the solution you'd like
This proposal is to add a basic conditional support for identifying what data is dropped by the DropEventsProcessor. A new drop_when configuration setting can be added following the Data Prepper Expression Syntax to specify what events will be dropped or continue to the next processor. The drop_when expression will evaluate once for each event. If the expression evaluates to true, the event will be dropped.

While evaluating an event if an exception occurs, by default, a warning will be logged and the event will not be dropped. Alternative behaviors can be configured with a new handle_failed_events configuration setting. The following options are supported:

Setting	Description	Data Prepper Version
skip	The event will not be dropped. A warning will be logged	1.3.0
skip_silently	The event will not be dropped. No warning will be logged	1.3.0
drop	The event will be dropped. A warning will be logged	1.3.0
drop_silently	The event will be dropped. No warning will be logged	1.3.0

Sample Pipeline Configuration

drop-pipeline:
  source:
    file:
      path: "/full/path/to/logs_json.log"
      record_type: "event"
      format: "json"
  processor:
    - drop_events:
        drop_when: '/status_code < 400'
        handle_failed_events: skip_silently
  sink:
    - stdout:

Describe alternatives you've considered (Optional)
Additional support for conditionals will be added by #1007

Additional context
Data Prepper Expression Parser
Data Prepper Expression Evaluator
Data Prepper Expression Syntax
Data Prepper Conditionals
Routers
Duplicate of closed issue #1063

[dlvenable]

Data Prepper may still add the when condition to processors as a core feature. Because of that, it may make sense to use a different name for when here. Perhaps drop_when?

I do like the options for exceptions. The naming should be consistent with other properties: on_when_exception, skip_silently, and drop_silently.

Putting it all together, is this what it might look like?

drop:
  drop_when: /status_code < 400
  on_when_exception: skip_silently

Also, do you think that handle_failed_events might be a clearer name than on_when_exception?

drop:
  drop_when: /status_code < 400
  handle_failed_events: skip_silently

[dlvenable]

I had already created #1063, FYI.

[sbayer55]

Thanks for the input @dlvenable. I agree drop_when and handle_failed_event is a more readable set of optionas.