Add decode base64 field processor

[mmatur]

Description

This PR adds a new processor to be able to decode base64.

The decode_base64_field processor decodes field containing Base64 strings.

processors:
 - decode_base64_field:
     field: "field1"
     target: ""

The decode_base64_field processor has the following configuration settings:

• field:: The field containing Base64 strings to decode.
• target:: (Optional) The field under which the decoded Base64 will be written. By
  default the decoded Base64 object replaces the string field from which it was
  read.
  target with an empty string (target: ""). Note that the null value (target:)
  is treated as if the field was not set at all.

[elasticmachine]

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

[kvch]

jenkins test this

[ph]

jenkins test this please

[kvch]

I think it would be good to make this configurable, just like in case of other processors. By adding fail_on_error and ignore_missing users could eliminate flooding the logs of Beats if the keys are not present in the event. See the implementation in rename processor: https://github.com/elastic/beats/blob/master/libbeat/processors/actions/rename.go#L36

[ph]

the code looks good, I am just worried about the usage and the configuration. The current code can work with a single instance of a processors on multiple fields.

I wonder if it's better to make it a 1:1 mapping, each field need to define his own processors. I am thinking about this because the errors handling looks stranges when it's applied on multiple fields.

@kvch in your comments #11914 (comment) were you asking only to add errors handling to the processors or allow a single processor to operate on multiple fields?

[andrewkroh]

I wonder if it's better to make it a 1:1 mapping, each field need to define his own processors. I am thinking about this because the errors handling looks stranges when it's applied on multiple fields.

So true. This really complicates things in other processors and I think having a 1:1 relationship like Ingest Node has would simplify many things.

[andrewkroh]

@mmatur Can you please also register this with the script processor so that it can be used there as well. Just need to add a line here:

beats/libbeat/processors/script/javascript/module/processor/processor.go

Lines 45 to 48 in 874e01f

	var constructors = map[string]processors.Constructor{
	"AddCloudMetadata": add_cloud_metadata.New,
	"AddDockerMetadata": add_docker_metadata.New,
	"AddFields": actions.CreateAddFields,

[kvch]

@ph Yes, I would let one base64 processor work on multiple fields as in case of other processors. I think confusion around which field lead to an error can be avoided by adding the key name to the error message. WDYT?
My only concern regarding 1:1 mapping is having an unnecessarily long list of processors in configurations.

[mmatur]

@kvch @ph @andrewkroh do I need to keep 1:1 mapping?

[kvch]

Keep it please.

[mmatur]

Thanks @kvch. My PR seems to be ready ;)

[ph]

Almost there 👍 , just added a note for the documentation and a behavior changes when a type assertion fails.

[ph]

Jenkins test this please

[kvch]

Thank you!

[ph]

Jenkins test this

[ph]

@kvch I've retriggerred jenkins build just to make sure.

[ph]

Thanks @mmatur for adding this :)