Grafana
- By Canonical Observability
Channel | Revision | Published | Runs on |
---|---|---|---|
latest/stable | 114 | 01 Aug 2024 | |
latest/candidate | 117 | 28 Jun 2024 | |
latest/beta | 117 | 21 Jun 2024 | |
latest/edge | 118 | 12 Aug 2024 | |
1.0/stable | 93 | 12 Dec 2023 | |
1.0/candidate | 93 | 22 Nov 2023 | |
1.0/beta | 93 | 22 Nov 2023 | |
1.0/edge | 93 | 22 Nov 2023 |
juju deploy grafana-k8s --channel edge
Deploy Kubernetes operators easily with Juju, the Universal Operator Lifecycle Manager. Need a Kubernetes cluster? Install MicroK8s to create a full CNCF-certified Kubernetes system in under 60 seconds.
Platform:
A common behavioural flaw we see in Grafana is that sometimes the Dashboards do not display information. We see this materialised in Panels that display legends such as “N/A” or “No data”.
Verify that Panel PromQL expression is valid
First of all we need to make sure that the PromQL
expression that is executed in a Panel with an N/A
or a No data
legend is valid.
In order to do that, we need to edit that Panel by clicking on Edit
option:
Once we are in the edition page, we see the PromQL expression below the graphic:
Sometimes a PromQL expression does not return data because there is no time series matching the labels we are filtering by. Keep in mind that in PromQL, labels are placed between { }
.
So let’s try executing the same query, but without filters. In order to do that we need to:
- Remove the labels inside
{ }
- Replace
$__rate_interval
variable with a Time duration value.
So, our modified query looks like this:
(sum by(instance) (irate(node_cpu_seconds_totals{}[5m])) / on(instance) group_left sum by (instance)((irate(node_cpu_seconds_total{}[5m])))) * 100
Now, let’s click on Menu/Explore
and execute modified query:
As we can see this query still does not return any results.
But if we look a little closer, we see that we are calculating the quotient of two sub-expressions that each have two different metrics: node_cpu_seconds_total
and node_cpu_seconds_totals
.
Confirm all metric names exists. If the metric (with the given labels) was not yet ingested by Prometheus, or if there is a typo in the metric name, then you will see “no data”. You can check if a metric exists by evaluating it in isolation in the “Explore” tab.
Bingo! we have found the error in our expression. Let’s go back to the Panel edition and fix the query there:
Verify that Drop-downs expressions are valid
Sometimes we have more than one panel that shows N/A
or No data
legends instead of a numerical value or a valid graphic.
In the header of our Dashboard, it is common to have drop-downs to apply filters to the dashboard graphics.
Let’s verify those drop-downs. To do that we have to click in the gear
in the header and go to the Variables
option:
Some of those variables
defined in the Dashboard can be used as drop-downs. Besides those variables can be of different types: comma separated values, data source, query, etc.
Often the error lies in the variables of the query type. As in the previous case, the expression does not return values either because we refer to a metric that does not exist, or because we apply a combination of labels that does not produce a result.
In our example we have an error in the name of the metric. It is node_uname_info
instead of node_uname_inf
If we apply that fix and go back to the Dashboard, we can see that the drop-downs have values and all graphics are working. Everything looks good!