Loki
- Canonical Observability
Channel | Revision | Published | Runs on |
---|---|---|---|
latest/stable | 160 | 10 Sep 2024 | |
latest/candidate | 161 | 19 Nov 2024 | |
latest/beta | 177 | 19 Nov 2024 | |
latest/edge | 181 | 30 Nov 2024 | |
1.0/stable | 104 | 12 Dec 2023 | |
1.0/candidate | 104 | 22 Nov 2023 | |
1.0/beta | 104 | 22 Nov 2023 | |
1.0/edge | 104 | 22 Nov 2023 |
juju deploy loki-k8s --channel 1.0/stable
Deploy Kubernetes operators easily with Juju, the Universal Operator Lifecycle Manager. Need a Kubernetes cluster? Install MicroK8s to create a full CNCF-certified Kubernetes system in under 60 seconds.
Platform:
Also see: How to instrument a charm for distributed tracing
Charm execution logging
When we talk about logging in Juju, we usually refer to logs emitted by the workload that a charm is operating. However, you can also collect the logs emitted by the charm code execution itself for debugging and troubleshooting purposes.
Fetch the charm_logging
charm library
To start, grab the charm_logging
lib:
charmcraft fetch-lib charms.loki_k8s.v0.charm_logging
The charm_logging
lib contains all you need to configure your charm to automatically forward any logs to a Loki instance over an existing loki_push_api
integration.
This howto assumes that your charm already has an integration to loki-k8s
over the loki_push_api
relation interface. See integrating with loki-k8s over loki_push_api
for instructions. The charm_logging
lib will use the same integration. This means that the same Loki instance that stores workload logs will also receive charm logs. In practice, you might want them to be separate Loki instances. To achieve that, add a separate integration to your charm and point charm_logging
to that one instead.
This means that, if your charm is related to loki-k8s
charm and loki-k8s
is related to grafana-k8s
over grafana-source
, you will be able to inspect in real time from the Grafana explorer the logs emitted by your charm.
Quickstart: using the charm_logging
library
The main entry point to the charm_logging
library is the log_charm
decorator.
Assuming you already have an integration over tracing
, and you’re already using lib.charms.loki_k8s.v1.loki_push_api.LokiPushApiConsumer
, you will need to:
- import
lib.charms.loki_k8s.v0.charm_logging.log_charm
- decorate your charm class (if you have multiple, only decorate the ‘last one’ i.e. the one you pass to
ops.main.main()
!) withlog_charm
. - pass to
log_charm
forward-references to:- the url of a loki-push-api endpoint
- [optional] absolute path to a CA certificate on the
charm
container disk
Caution should be exercised in that:
- if the
loki_push_api_endpoint
you pass to the decorator is notNone
, charm logging will be enabled - if charm logging is enabled, and the endpoint is HTTPS, but no
cert_path
is passed to the decorator, (or, the path is invalid/does not exist), you will get errors.
To help generate the right configuration and avoid inconsistent situations, loki_push_api
comes with a charm_logging_config
helper to determine when to disable charm logging because the endpoint is HTTPS but we don’t have a certificate on disk yet. The charm is still responsible for writing the cert file to disk as soon as one is available.
For example:
from lib.charms.loki_k8s.v0.charm_logging import log_charm
from lib.charms.loki_k8s.v1.loki_push_api import LokiPushApiConsumer, charm_logging_config
@log_charm(loki_push_api_endpoint="my_logging_endpoint", cert_path="cert_path")
class MyCharm(...):
# if your charm has an integration to a CA certificate provider, you can copy the CA certificate on the charm container and use it to encrypt logs sent to Loki
_cert_path = "/path/to/cert/on/charm/container/cacert.crt"
def __init__(self, ...):
# the tracing integration
self.logging = LokiPushApiConsumer(self, ...)
# this data will be picked up by the decorator and used to determine where to send the logs to, and what CA cert (if any) to use to encrypt them if the endpoint is https.
self.my_logging_endpoint, self.cert_path = charm_logging_config(
self.logging, self._cert_path)
At this point any logs emitted throughout the lifecycle of MyCharm
are automatically forwarded to Loki. All that you will see in the juju debug-log
is now available in grafana.
Future work
- Associating charm traces to logs and vice versa (exemplars) in grafana