Grafana

  • By Canonical Observability
Channel Revision Published Runs on
latest/stable 114 28 Jun 2024
Ubuntu 20.04
latest/candidate 117 28 Jun 2024
Ubuntu 20.04
latest/beta 117 21 Jun 2024
Ubuntu 20.04
latest/edge 117 18 Jun 2024
Ubuntu 20.04
1.0/stable 93 12 Dec 2023
Ubuntu 20.04
1.0/candidate 93 22 Nov 2023
Ubuntu 20.04
1.0/beta 93 22 Nov 2023
Ubuntu 20.04
1.0/edge 93 22 Nov 2023
Ubuntu 20.04
juju deploy grafana-k8s
Show information

Platform:

A common behavioural flaw we see in Grafana is that sometimes the Dashboards do not display information. We see this materialised in Panels that display legends such as “N/A” or “No data”.

image

Verify that Panel PromQL expression is valid

First of all we need to make sure that the PromQL expression that is executed in a Panel with an N/A or a No data legend is valid.

In order to do that, we need to edit that Panel by clicking on Edit option:

image

Once we are in the edition page, we see the PromQL expression below the graphic:

Sometimes a PromQL expression does not return data because there is no time series matching the labels we are filtering by. Keep in mind that in PromQL, labels are placed between { }.

So let’s try executing the same query, but without filters. In order to do that we need to:

  • Remove the labels inside { }
  • Replace $__rate_interval variable with a Time duration value.

So, our modified query looks like this:

(sum by(instance) (irate(node_cpu_seconds_totals{}[5m])) / on(instance) group_left sum by (instance)((irate(node_cpu_seconds_total{}[5m])))) * 100

Now, let’s click on Menu/Explore

image

and execute modified query:

As we can see this query still does not return any results.

But if we look a little closer, we see that we are calculating the quotient of two sub-expressions that each have two different metrics: node_cpu_seconds_total and node_cpu_seconds_totals.

Confirm all metric names exists. If the metric (with the given labels) was not yet ingested by Prometheus, or if there is a typo in the metric name, then you will see “no data”. You can check if a metric exists by evaluating it in isolation in the “Explore” tab.

Bingo! we have found the error in our expression. Let’s go back to the Panel edition and fix the query there:

Verify that Drop-downs expressions are valid

Sometimes we have more than one panel that shows N/A or No data legends instead of a numerical value or a valid graphic.

In the header of our Dashboard, it is common to have drop-downs to apply filters to the dashboard graphics.

Let’s verify those drop-downs. To do that we have to click in the gear in the header and go to the Variables option:

Some of those variables defined in the Dashboard can be used as drop-downs. Besides those variables can be of different types: comma separated values, data source, query, etc.

Often the error lies in the variables of the query type. As in the previous case, the expression does not return values either because we refer to a metric that does not exist, or because we apply a combination of labels that does not produce a result.

In our example we have an error in the name of the metric. It is node_uname_info instead of node_uname_inf

If we apply that fix and go back to the Dashboard, we can see that the drop-downs have values and all graphics are working. Everything looks good!