Grafana Agent

  • Canonical Observability
Channel Revision Published Runs on
latest/stable 75 04 Jun 2024
Ubuntu 22.04 Ubuntu 20.04
latest/stable 45 12 Dec 2023
Ubuntu 22.04 Ubuntu 20.04
latest/candidate 80 01 Aug 2024
Ubuntu 22.04 Ubuntu 20.04
latest/candidate 75 04 Jun 2024
Ubuntu 22.04 Ubuntu 20.04
latest/candidate 45 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
latest/beta 75 22 May 2024
Ubuntu 22.04 Ubuntu 20.04
latest/beta 45 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
latest/edge 97 14 Nov 2024
Ubuntu 22.04 Ubuntu 20.04
latest/edge 75 03 May 2024
Ubuntu 22.04 Ubuntu 20.04
latest/edge 45 15 Sep 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/stable 51 12 Dec 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/stable 45 12 Dec 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/candidate 51 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/candidate 45 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/beta 51 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/beta 45 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/edge 51 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
1.0/edge 45 22 Nov 2023
Ubuntu 22.04 Ubuntu 20.04
juju deploy grafana-agent-k8s --channel beta
Show information

Platform:

Relations

Overview

Relations provide a means to integrate applications and enable a simple communications channel. Grafana Agent Charmed Operator supports the following:

Provides

Logging provider

  logging-provider:
    interface: loki_push_api

Grafana Agent Charmed Operator may receive logs from any charm that supports the loki_push_api relation interface.

The information exchanged through the loki_push_api interface can be broken down into two parts:

  • Grafana Agent charm provides an endpoint URL to receive log from Loki clients that relates with this charm.
  • Grafana Agent charm may also receive alert rules which tell when to raise alerts. These rules are read from a directory named loki_alert_rules, if present at the top level, within the client charm’s source (src) directory.

For instance let’s say that we have a Zinc Charmed Operator that implements the other side (requires) of the relation. After deploying this charm, we can relate Grafana Agent and Zinc through loki_push_api relation interface:

juju relate grafana-agent-k8s:logging-provider zinc-k8s

Once the relation is established, Zinc charm can start sending logs to Grafana Agent charm.

Grafana dashboard

  grafana-dashboard:
    interface: grafana_dashboard

Over the grafana-dashboard relation using the grafana-dashboard interface, this Grafana Agent charm also provides meaningful dashboards about its metrics to be shown in a Grafana Charm .

In order to add these dashboards to Grafana all that is required is to relate the two charms in the following way:

juju relate \
    grafana-agent-k8s:grafana-dashboard \
    grafana-k8s:grafana-dashboard

Self metrics endpoint

self-metrics-endpoint:
    interface: prometheus_scrape

This Grafana Agent charm may forward information about its metrics endpoint and associated alert rules to a Prometheus charm over the self-metrics-endpoint relation using the prometheus_scrape interface. In order for these metrics to be aggregated by the remote Prometheus charm all that is required is to relate the two charms as in:

juju relate \
    grafana-agent-k8s:self-metrics-endpoint \
    prometheus-k8s:metrics-endpoint

Requires

Logging consumer

  logging-consumer:
    interface: loki_push_api

Loki receives logs forwarded from Grafana Agent, aggregates and deduplicates them. Grafana Agent Charmed Operator relates to Loki over the loki_push_api interface using the logging-consumer relation.

juju relate grafana-agent-k8s:logging-consumer loki-k8s

Metrics endpoint

  metrics-endpoint:
    interface: prometheus_scrape

Charms may forward information about their metrics endpoints and associated alert rules to the Grafana Agent charm over the metrics-endpoint relation using the prometheus_scrape interface.

juju relate grafana-agent-k8s:metrics-endpoint zinc-k8s

Charms that seek to provide metrics endpoints and alert rules must do so using the provided prometheus_scrape charm library. This library by implementing the metrics-endpoint relation, not only ensures that scrape jobs and alert rules are forward to Grafana Agent but also that these are updated any time the metrics provider charm is upgraded. For example new alert rules may be added or old ones removed by updating and releasing a new version of the metrics provider charm. While it is safe to update alert rules as desired, care must be taken when updating scrape job specifications as this has the potential to break the continuity of the scraped metrics time series. In particular changing the following keys in the scrape job can break time series continuity

  • job_name
  • relabel_configs
  • metrics_relabel_configs
  • Any label set by static_configs

Evaluation of alert rules forwarded through the prometheus_scrape interface are automatically limited to the charmed application that provided these rules. This ensures that alert rule evaluation is scoped down to the charm providing the rules.

Send remote write

  send-remote-write:
    interface: prometheus_remote_write

Grafana Agent may forward client charms metrics and associated alert rules (that are received using the metrics-endpoint relation) to Prometheus using the prometheus_remote_write interface. To do that you need to relate Grafana Agent with Prometheus:

juju relate grafana-agent-k8s:send-remote-write prometheus-k8s

Once these relations are established, you can check that by running:

juju status --relations

Model  Controller  Cloud/Region        Version  SLA          Timestamp
docs   charm-dev   microk8s/localhost  2.9.32   unsupported  17:55:18-03:00

App                Version  Status   Scale  Charm              Channel    Rev  Address         Exposed  Message
grafana                     active       1  grafana-k8s        edge        40  10.152.183.168  no
grafana-agent-k8s           active       1  grafana-agent-k8s  edge        11  10.152.183.123  no
loki-k8s                    waiting      1  loki-k8s           edge        41  10.152.183.43   no
prometheus-k8s     2.33.5   active       1  prometheus-k8s     candidate   55  10.152.183.128  no
zinc-k8s           0.2.8    active       1  zinc-k8s           stable      26  10.152.183.247  no

Unit                  Workload  Agent       Address       Ports  Message
grafana-agent-k8s/0*  active    idle        10.1.157.120
grafana/0*            active    idle        10.1.157.88
loki-k8s/0            active    idle        10.1.157.86
prometheus-k8s/0*     active    idle        10.1.157.89
zinc-k8s/0*           active    idle        10.1.157.79

Relation provider                        Requirer                             Interface                Type     Message
grafana-agent-k8s:grafana-dashboard      grafana:grafana-dashboard            grafana_dashboard        regular
grafana-agent-k8s:logging-provider       zinc-k8s:logging                     loki_push_api            regular
grafana-agent-k8s:self-metrics-endpoint  prometheus-k8s:metrics-endpoint      prometheus_scrape        regular
grafana:grafana                          grafana:grafana                      grafana_peers            peer
loki-k8s:logging                         grafana-agent-k8s:logging-consumer   loki_push_api            regular
prometheus-k8s:prometheus-peers          prometheus-k8s:prometheus-peers      prometheus_peers         peer
prometheus-k8s:receive-remote-write      grafana-agent-k8s:send-remote-write  prometheus_remote_write  regular
zinc-k8s:metrics-endpoint                grafana-agent-k8s:metrics-endpoint   prometheus_scrape        regular