Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Grafana Loki destination #4631

Merged
merged 11 commits into from
Sep 22, 2023
Merged

Grafana Loki destination #4631

merged 11 commits into from
Sep 22, 2023

Conversation

MrAnno
Copy link
Collaborator

@MrAnno MrAnno commented Sep 11, 2023 •
edited
Loading

This PR implements the gRPC-based loki() destination.

The message format is the same as in their documented HTTP endpoint:
https://grafana.com/docs/loki/latest/reference/api/#push-log-entries-to-loki

Example config:

loki(
    url("localhost:9096")
    # timestamp(msg)
    labels(
        "app" => "$PROGRAM",
        "host" => "$HOST",
    )

    workers(16)
    batch-timeout(10000)
    batch-lines(1000)
);

Depends on #4630

@MrAnno MrAnno force-pushed the loki branch 2 times, most recently from 8b6d361 to c95e551 Compare September 11, 2023 00:18
@kira-syslogng
Copy link
Contributor

Build FAILURE

@MrAnno MrAnno added this to the syslog-ng 4.4 milestone Sep 19, 2023
@MrAnno MrAnno marked this pull request as ready for review September 20, 2023 10:16
@MrAnno MrAnno force-pushed the loki branch 2 times, most recently from a819ff8 to c9c183a Compare September 20, 2023 10:38
@MrAnno
Copy link
Collaborator Author

MrAnno commented Sep 20, 2023

@kira-syslogng do stresstest

@kira-syslogng
Copy link
Contributor

Kira-stress-test: Build SUCCESS

@MrAnno MrAnno requested a review from alltilla September 20, 2023 15:25
@MrAnno
grpc: add comment about how to avoid adding built protobufs to the ta…
c99f940 
…rball

Signed-off-by: László Várady <[email protected]>
@MrAnno
grpc: arrange files in alphabetical order
b99addf 
Signed-off-by: László Várady <[email protected]>
@MrAnno
loki: add skeleton destination
6d4db9f 
Signed-off-by: László Várady <[email protected]>
@MrAnno
copyright: update policy with loki and otel plugins
e7e3c49 
Signed-off-by: László Várady <[email protected]>
@MrAnno
packaging: add Loki plugin
632b1aa 
Signed-off-by: László Várady <[email protected]>
@MrAnno
loki: implement Loki ingestion
0f8826c 
Signed-off-by: László Várady <[email protected]>
@MrAnno
loki: add timestamp(current|received|msg) option
5fca19d 
Loki requires monotonic timestamps within the same label-set, which makes
it difficult to use the original message timestamp without the possibility
of message loss. In case the monotonic property is violated, Loki discards
the problematic messages with an error:

> entry too far behind, oldest acceptable timestamp is: ...

For this reason, the default value is `current`.

Signed-off-by: László Várady <[email protected]>
@alltilla alltilla merged commit a8fea54 into syslog-ng:master Sep 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alltilla alltilla alltilla approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
syslog-ng 4.4
Development

Successfully merging this pull request may close these issues.
3 participants
@MrAnno @kira-syslogng @alltilla