Create a new dashboard for an existing charm

Introduction

By the end of this tutorial you will have a grafana dashboard for self-monitoring prometheus, with juju topology drop-downs.

Deploy the apps

First, deploy grafana and prometheus:

juju deploy --trust --channel=edge grafana-k8s grafana
juju deploy --trust --channel=edge prometheus-k8s prometheus

Next, relate the two charms so grafana could see some metrics:

juju relate prom grafana:metrics-endpoint
juju relate prom:grafana-source grafana                 
juju relate prom:grafana-dashboard grafana

Open grafana’s dashboard creation web ui

First, obtain the admin password:

juju run grafana/0 get-admin-password

Next

1. Navigate with your browser to grafana’s unit ip, and log in with “admin”, and the password you just obtained.
2. Familiarize yourself with the dashboard creation process from the official docs and head over to http://GRAFANA_UNIT_IP:3000/dashboard/new.

Create a new dashboard for prometheus

1. Add a new panel and set juju_..._prometheus_0 as the data source.
2. Switch from “Builder” to “Code”.
3. In the PromQL query, type go_threads and press Shift+Enter. You should see two lines, one for prometheus and one for grafana itself (grafana will be automatically filtered-out in a later step). Click “Apply”.
4. Add another panel with the following query: sum by (handler) (rate(prometheus_http_request_duration_seconds_sum[$__rate_interval])). Switch the visualization type to “Heatmap”. Click “Apply”.
5. Save the dashboard.

Export dashboard

1. Click “Share dashboard or panel” at the top left.
2. Go to “Export” tab and deselect Export for sharing externally. Save to file.

Before proceeding to the next steps, delete the dashboard from grafana to avoid collision.

Manually modify the exported dashboard

First, replace all datasource entries in the dashboard json file with ${prometheusds}.

For grafana versions older than 9 it should look like this:

"datasource": "${prometheusds}",

For recent grafana versions it should look like this:

      "datasource": {
        "type": "prometheus",
        "uid": "${prometheusds}"
      },

Next, add juju topology label selectors to all expressions, escaping the quotation marks. For example:

"expr": "go_threads{juju_model_uuid=~\"$juju_model_uuid\", juju_model=~\"$juju_model\",juju_application=~\"$juju_application\", juju_unit=~\"$juju_unit\"}"

and

"expr": sum by (handler) (rate(prometheus_http_request_duration_seconds_sum{juju_model_uuid=~\"$juju_model_uuid\", juju_model=~\"$juju_model\",juju_application=~\"$juju_application\", juju_unit=~\"$juju_unit\"}[$__rate_interval]))

Add dashboard to the prometheus charm

To include the exported dashboard in the collection of built-in prometheus dashboards:

1. Place the dashboard under the grafana_dashboards folder, pack and refresh the charm.
2. In the browser, refresh the list of dashboards, http://GRAFANA_UNIT_IP:3000/dashboards.
3. In your new dashboard you should now see selectable drop-down menus that affect the content displayed on the graphs.