Prometheus

  • Canonical Observability
Channel Revision Published Runs on
latest/stable 210 19 Nov 2024
Ubuntu 20.04
latest/candidate 216 Yesterday
Ubuntu 20.04
latest/beta 221 Yesterday
Ubuntu 20.04
latest/edge 221 03 Dec 2024
Ubuntu 20.04
1.0/stable 159 16 Feb 2024
Ubuntu 20.04
1.0/candidate 159 12 Dec 2023
Ubuntu 20.04
1.0/beta 159 12 Dec 2023
Ubuntu 20.04
1.0/edge 159 12 Dec 2023
Ubuntu 20.04
juju deploy prometheus-k8s
Show information

Platform:

Mimir provides a scalable and long-term storage solution for metrics, making it a suitable alternative for Prometheus. This How-to will highlight validation steps to provide confidence to the operator that the migration from Prometheus to Mimir is successful without data loss.

Deploy Mimir

Deploy Mimir alongside Prometheus. Then relate Mimir to all the same applications and endpoints that Prometheus has.

Validation

Now we can validate whether Mimir is receiving the same data as Prometheus.

These validations are not exhaustive. More validations can be added to boost migration confidence

Validate metric name outputs

  • juju ssh prometheus/0 curl -g 'http://localhost:9090/api/v1/label/__name__/values' | jq -r ".data[]" | sort -u > prom-metric.txt
  • juju ssh mimir/0 curl -g 'http://localhost:8080/prometheus/api/v1/label/__name__/values' | jq -r ".data[]" | sort -u > mimir-metric-names.txt

Confirm that the same metrics in Prometheus are in Mimir by inspecting diff:

diff -y prom-metrics.txt mimir-metrics.txt

Other validations

Validation Prometheus Mimir
labels api/v1/labels /prometheus/api/v1/labels
app names api/v1/label/juju_application/values prometheus/api/v1/label/juju_application/values
alerts api/v1/rules prometheus/api/v1/rules

For alerts, the API response will result in a large JSON output and we can modify the jq to jq -r ".data[][].name" to aggregate the alert names for validation.

Remove Prometheus

Once the validations are complete, check the config value for metrics_retention_time with the command:

juju config prometheus metrics_retention_time

This is how long samples are retained in the storage (Units Supported: y, w, d, h, m, s). Before deleting Prometheus, this much time must pass prior to removing the Prometheus application to ensure that no data is lost.

Once Prometheus is removed, you are done and you can enjoy a modernized deployment.