feature: support OpenTelemetry Protocol (OTLP)

[connec]

I'm very interested in using parseable as it looks like a great answer to bloated/closed logging services, but I'm also keen to adopt OTLP for my projects in the hope that they could then be integrated seamlessly into different environments with different logging services (including, tragically, bloated/closed ones).

Is support for the OpenTelemetry Protocol (OTLP, specification) on the roadmap at all? Would there be appetite to discuss it/merge a PR introducing it?

[nitisht]

hi @connec thank you for the issue. We’re interested in OTEL support. I am curious what would it entail from Parseable server perspective, AFAICS a client can send OTLP logs and we can ingest that (since Parseable is schema free).

Happy to discuss here or on our Slack.

[connec]

You may well be right that JSON over HTTP is already supported! I'm a bit of a neophyte in this area, but it looks like JSON over HTTP is still "experimental" in the OTLP spec, and that grpc/protobuf over HTTP are the stable options, so I assumed a dedicated endpoint would be needed.

I'd understand if you'd rather keep the interface to JSON/HTTP, and I can instead look at the exporter I'm using and adding JSON/http support there.

[nitisht]

it looks like JSON over HTTP is still "experimental" in the OTLP spec, and that grpc/protobuf over HTTP are the stable options

ah, I missed the experimental part. It makes sense for Parseable to have grpc support for ingestion. Would you be interested in a contributing dedicated endpoint for grpc?

[shubhwip]

Hi @nitisht I would like to contribute to this feature.

[nitisht]

Hey @shubhwip thanks for the offer. Would you join our slack at https://launchpass.com/parseable and we can discuss further?

[masenocturnal]

I'm also looking at this and would love to be part of the conversation. I've done a lot of work in generating optimal parquet from otlp payloads

[masenocturnal]

The oltp collector also has a branch which uses apache arrow over the wire.

[nitisht]

The oltp collector also has a branch which uses apache arrow over the wire.

Thanks @masenocturnal this is an interesting idea. we'll prioritise this issue again. This had fallen through the cracks.