---
title: Charmhub | Deploy Grafana Agent using Charmhub - The Open Operator Collection
description: Deploy the latest version of Grafana Agent as a Kubernetes Operator on
  any cloud.
url: https://charmhub.io/grafana-agent-k8s/docs/uncharmed
---

# Grafana Agent

[Canonical Observability](https://charmhub.io/publisher/observability "View all packages from Canonical Observability")

* [Canonical Observability](https://charmhub.io/publisher/observability "View all packages from Canonical Observability")

Platform:

0.40/stable rev229

```
juju deploy grafana-agent-k8s --channel 0.40/stable
```

[Learn to deploy on juju >](https://juju.is/docs/juju/manage-applications)

---

#### Relevant links

* [Homepage](https://charmhub.io/grafana-agent-k8s)

---

#### Contacts

* [Submit a bug](https://github.com/canonical/grafana-agent-k8s-operator/issues)

---

Share your thoughts on this charm with the community on discourse.

[Join the discussion](https://discourse.charmhub.io/)

The [`cos-lite` bundle](https://github.com/canonical/cos-lite-bundle) is meant to be run by Juju. However, not all workloads that you may want to monitor do. The good news is that you can use `cos-lite` to monitor workloads that are not charmed (aka ‘not managed by Juju’). The bad news is that it’s relatively straightforward to do so. Not bad at all.

**Contents:**

* [Deploy `cos-lite`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--deploy-cos-lite)
* [Deploy `grafana-agent`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--deploy-grafana-agent)
* [Get the API endpoints from `traefik`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--get-the-api-endpoints-from-traefik)
* [Add custom dashboards and alerts](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--add-custom-dashboards-and-alerts)
* [TLS](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--tls)
* [Known limitations and upcoming features](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--known-limitations-and-upcoming-features)
  + [Identity](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--identity)
  + [Tracing](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--tracing)
* [Only export metrics with `prometheus-scrape-target`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--only-export-metrics-with-prometheus-scrape-target)

## [Deploy `cos-lite`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--deploy-cos-lite)

The first step will be to get a hold of a machine, somewhere, and follow [this guide on how to get started with COS lite on microk8s](https://charmhub.io/topics/canonical-observability-stack/tutorials/install-microk8s).

And be sure to follow the [best practices](https://discourse.charmhub.io/t/12012)!

Unless you’re also planning to monitor some charmed applications with this cos-lite deployment, you will not need to use [the `offers` overlay](https://charmhub.io/topics/canonical-observability-stack/tutorials/install-microk8s#heading--deploy-the-cos-lite-bundle-with-overlays).

## [Deploy `grafana-agent`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--deploy-grafana-agent)

The Grafana agent will act as an intermediary between the applications you want to monitor and the `cos-lite` stack. It will gather telemetry from your applications and send them to `cos-lite`, where you will be able to inspect them through the Grafana dashboards.

We recommend to host the Grafana agent as close as possible to the workloads you intend to monitor, to minimise the risk of network faults and the resulting gaps in telemetry collection.
Also, we recommend to install the Grafana agent via a handy snap we maintain:

However, Grafana agent is also available as a single Go binary, and you are free to install it and run it the way you like. See the [official documentation](https://grafana.com/docs/agent/latest/) for the publisher’s recommendations and guides.

Last but not least, we also have it [containerized](http://ghcr.io/canonical/grafana-agent) and [petrified](https://github.com/canonical/grafana-agent-rock/).

Now that you have Grafana Agent up and running, you will need to configure it.

## [Get the API endpoints from `traefik`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--get-the-api-endpoints-from-traefik)

`cos-lite` includes a `traefik` instance that takes care of load balancing and ingressing the various observability components of the stack. Since `cos-lite` runs on Kubernetes, this allows you to talk to them via `traefik` over a stable URL.

Before you can use Traefik from an external service such as Grafana agent, you will need to ensure that the Traefik URL is routable from the service host, and that the address is stable. (e.g. not a dynamic IP)
In other words, Traefik’s own URL aso needs to be stable.

In the Juju model where `cos-lite` is installed, you can run:

> `juju run traefik/0 show-proxied-endpoints`

Assuming you have [configured the `traefik` charm](https://github.com/canonical/traefik-k8s-operator#configurations) to use an external hostname, for example `"traefik.url"`, you will see something like:

```
proxied-endpoints: '{
    "prometheus/0": {"url": "https://traefik.url/mymodel-prometheus-0"},
    "loki/0": {"url": "https://traefik.url/mymodel-loki-0"},
    "alertmanager": {"url": "https://traefik.url/mymodel-alertmanager"},
    "catalogue": {"url": "https://traefik.url/mymodel-catalogue"},
}'
```

You can also open `https://traefik.url/mymodel-catalogue` in a browser to see a page with links to all `cos-lite` components’ user interfaces.

At this point you will need to follow [the documentation on how to configure the Grafana agent](https://grafana.com/docs/agent/latest/static/configuration/#configure-static-mode).
Use the urls you obtained from traefik to tell the agent where to send its telemetry.

## [Add custom dashboards and alerts](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--add-custom-dashboards-and-alerts)

In order to add your own dashboards and alerts to `cos-lite` you will need to deploy the [`cos-config` charm](https://github.com/canonical/cos-configuration-k8s-operator) on top of `cos-lite`.

Follow [this guide](https://github.com/canonical/cos-configuration-k8s-operator#deployment) to set up `cos-config` in the same Juju model in which `cos-lite` is deployed.

## [TLS](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--tls)

To enable secure communications with (and within) COS Lite, deploy cos-lite with the [tls overlay](https://github.com/canonical/cos-lite-bundle/pull/80) by follow [this guide](https://charmhub.io/traefik-k8s/docs/tls-termination) to enable TLS in Traefik and COS Lite.

### [Grafana-agent as a client](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#p-27862-grafana-agent-as-a-client)

As a client, grafana-agent must trust the CA that signed the COS charms (or the COS ingress charm).

For example, to obtain the ca-cert from the [self-signed-certificates](https://charmhub.io/self-signed-certificates) charm,

```
juju run ssc/0 get-ca-certificate --format=yaml \
  | yq '.ssc/0.results.ca-certificate'
```

Next, you need to [add the certificate to the root store](https://documentation.ubuntu.com/server/how-to/security/install-a-root-ca-certificate-in-the-trust-store/index.html).

> Note: After running `update-ca-certificates` and restarting the grafana-agent service, check the grafana-agent logs to confirm there are no log lines such as:
>
> `msg="Failed to send batch, retrying" err="Post \"https://.../api/v1/write\": tls: failed to verify certificate: x509: certificate signed by unknown authority"`

### [Grafana-agent as a server](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#p-27862-grafana-agent-as-a-server)

To have a TLS handshake with incoming connections, you need to have a private key and a cert [stored](https://snapcraft.io/docs/data-locations) (preferably in [`$SNAP_USER_COMMON`](https://snapcraft.io/docs/environment-variables#heading--snap-user-common)), and add to the grafana agent config as follows:

```
server:
  log_level: info
+ grpc_tls_config:
+   cert_file: /home/ubuntu/snap/grafana-agent/common/grafana-agent.pem
+   key_file: /home/ubuntu/snap/grafana-agent/common/grafana-agent.key
+ http_tls_config:
+   cert_file: /home/ubuntu/snap/grafana-agent/common/grafana-agent.pem
+   key_file: /home/ubuntu/snap/grafana-agent/common/grafana-agent.key
```

## [Known limitations and upcoming features](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--known-limitations-and-upcoming-features)

### [Identity](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--identity)

We are “working towards”[citation needed] an integration with canonical’s [IAM bundle](https://github.com/canonical/iam-bundle) to provide a charmed identity solution to support locking down your observability stack behind an identity provider. Stay tuned for updates!

### [Tracing](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--tracing)

We are “working towards”[citation needed] a [tracing overlay](https://github.com/canonical/cos-lite-bundle/pull/79) to add distributed tracing capabilities to cos-lite. Once that work is done, you will be able to add [Grafana Tempo](https://grafana.com/oss/tempo/) to the stack.

## [Only export metrics with `prometheus-scrape-target`](https://charmhub.io/grafana-agent-k8s/docs/uncharmed#heading--only-export-metrics-with-prometheus-scrape-target)

In some rare circumstances, you might prefer to use `prometheus-scrape-target` instead of `grafana-agent`.
Namely:

* when you only need metrics (no logs, traces, etc…)
* when you’d rather make the necessary firewall changes in the workload you want to monitor, than ingress cos-lite
* when you’re not able to install anything (or the grafana-agent anyway) on the workload you want to monitor

If this is your situation, we’ve got you covered. You can deploy [`prometheus-scrape-target`](https://github.com/canonical/prometheus-scrape-target-k8s-operator) and configure it to scrape your workload.

---