Tempo
- By Canonical Observability
Channel | Revision | Published | Runs on |
---|---|---|---|
latest/stable | 71 | 10 Sep 2024 | |
latest/stable | 9 | 16 Feb 2024 | |
latest/candidate | 75 | 10 Sep 2024 | |
latest/candidate | 9 | 12 Dec 2023 | |
latest/beta | 75 | 01 Aug 2024 | |
latest/beta | 9 | 12 Dec 2023 | |
latest/edge | 81 | 11 Sep 2024 | |
latest/edge | 9 | 31 Jul 2023 |
juju deploy tempo-k8s
Deploy Kubernetes operators easily with Juju, the Universal Operator Lifecycle Manager. Need a Kubernetes cluster? Install MicroK8s to create a full CNCF-certified Kubernetes system in under 60 seconds.
Platform:
charms.tempo_k8s.v1.charm_tracing
-
- Last updated 16 Aug 2024
- Revision Library version 1.15
This charm library contains utilities to instrument your Charm with opentelemetry tracing data collection.
(yes! charm code, not workload code!)
This means that, if your charm is related to, for example, COS' Tempo charm, you will be able to inspect in real time from the Grafana dashboard the execution flow of your charm.
Quickstart
Fetch the following charm libs (and ensure the minimum version/revision numbers are satisfied):
charmcraft fetch-lib charms.tempo_k8s.v2.tracing # >= 1.10
charmcraft fetch-lib charms.tempo_k8s.v1.charm_tracing # >= 2.7
Then edit your charm code to include:
# import the necessary charm libs
from charms.tempo_k8s.v2.tracing import TracingEndpointRequirer, charm_tracing_config
from charms.tempo_k8s.v1.charm_tracing import charm_tracing
# decorate your charm class with charm_tracing:
@charm_tracing(
# forward-declare the instance attributes that the instrumentor will look up to obtain the
# tempo endpoint and server certificate
tracing_endpoint="tracing_endpoint",
server_cert="server_cert"
)
class MyCharm(CharmBase):
_path_to_cert = "/path/to/cert.crt"
# path to cert file **in the charm container**. Its presence will be used to determine whether
# the charm is ready to use tls for encrypting charm traces. If your charm does not support tls,
# you can ignore this and pass None to charm_tracing_config.
# If you do support TLS, you'll need to make sure that the server cert is copied to this location
# and kept up to date so the instrumentor can use it.
def __init__(self, ...):
...
self.tracing = TracingEndpointRequirer(self, ...)
self.tracing_endpoint, self.server_cert = charm_tracing_config(self.tracing, self._path_to_cert)
Detailed usage
To use this library, you need to do two things:
- decorate your charm class with
@trace_charm(tracing_endpoint="my_tracing_endpoint")
- add to your charm a "my_tracing_endpoint" (you can name this attribute whatever you like)
property, method or instance attribute that returns an otlp http/https endpoint url.
If you are using the
charms.tempo_k8s.v2.tracing.TracingEndpointRequirer
asself.tracing = TracingEndpointRequirer(self)
, the implementation could be:
@property
def my_tracing_endpoint(self) -> Optional[str]:
'''Tempo endpoint for charm tracing'''
if self.tracing.is_ready():
return self.tracing.get_endpoint("otlp_http")
else:
return None
At this point your charm will be automatically instrumented so that:
- charm execution starts a trace, containing
- every event as a span (including custom events)
- every charm method call (except dunders) as a span
TLS support
If your charm integrates with a TLS provider which is also trusted by the tracing provider (the Tempo charm),
you can configure charm_tracing
to use TLS by passing a server_cert
parameter to the decorator.
If your charm is not trusting the same CA as the Tempo endpoint it is sending traces to, you'll need to implement a cert-transfer relation to obtain the CA certificate from the same CA that Tempo is using.
For example:
from charms.tempo_k8s.v1.charm_tracing import trace_charm
@trace_charm(
tracing_endpoint="my_tracing_endpoint",
server_cert="_server_cert"
)
class MyCharm(CharmBase):
self._server_cert = "/path/to/server.crt"
...
def on_tls_changed(self, e) -> Optional[str]:
# update the server cert on the charm container for charm tracing
Path(self._server_cert).write_text(self.get_server_cert())
def on_tls_broken(self, e) -> Optional[str]:
# remove the server cert so charm_tracing won't try to use tls anymore
Path(self._server_cert).unlink()
More fine-grained manual instrumentation
if you wish to add more spans to the trace, you can do so by getting a hold of the tracer like so:
import opentelemetry
...
def get_tracer(self) -> opentelemetry.trace.Tracer:
return opentelemetry.trace.get_tracer(type(self).__name__)
By default, the tracer is named after the charm type. If you wish to override that, you can pass
a different service_name
argument to trace_charm
.
See the official opentelemetry Python SDK documentation for usage: https://opentelemetry-python.readthedocs.io/en/latest/
Upgrading from v0
If you are upgrading from charm_tracing
v0, you need to take the following steps (assuming you already
have the newest version of the library in your charm):
- If you need the dependency for your tests, add the following dependency to your charm project
(or, if your project had a dependency on
opentelemetry-exporter-otlp-proto-grpc
only because ofcharm_tracing
v0, you can replace it with):
opentelemetry-exporter-otlp-proto-http>=1.21.0
.
- Update the charm method referenced to from
@trace
and@trace_charm
, to return fromTracingEndpointRequirer.get_endpoint("otlp_http")
instead ofgrpc_http
. For example:
from charms.tempo_k8s.v0.charm_tracing import trace_charm
@trace_charm(
tracing_endpoint="my_tracing_endpoint",
)
class MyCharm(CharmBase):
...
@property
def my_tracing_endpoint(self) -> Optional[str]:
'''Tempo endpoint for charm tracing'''
if self.tracing.is_ready():
return self.tracing.otlp_grpc_endpoint() # OLD API, DEPRECATED.
else:
return None
needs to be replaced with:
from charms.tempo_k8s.v1.charm_tracing import trace_charm
@trace_charm(
tracing_endpoint="my_tracing_endpoint",
)
class MyCharm(CharmBase):
...
@property
def my_tracing_endpoint(self) -> Optional[str]:
'''Tempo endpoint for charm tracing'''
if self.tracing.is_ready():
return self.tracing.get_endpoint("otlp_http") # NEW API, use this.
else:
return None
- If you were passing a certificate (str) using
server_cert
, you need to change it to provide an absolute path to the certificate file instead.
def is_enabled()
Description
Whether charm tracing is enabled. None
def charm_tracing_disabled()
Contextmanager to temporarily disable charm tracing.
Description
For usage in tests.
def get_current_span()
Return the currently active Span, if there is one, else None.
Description
If you'd rather keep your logic unconditional, you can use opentelemetry.trace.get_current_span, which will return an object that behaves like a span but records no data.
class TracingError
Description
Base class for errors raised by this module. None
class UntraceableObjectError
Description
Raised when an object you're attempting to instrument cannot be autoinstrumented. None
class TLSError
Description
Raised when the tracing endpoint is https but we don't have a cert yet. None
def
trace_charm(
tracing_endpoint: str,
server_cert,
service_name,
extra_types
)
Autoinstrument the decorated charm with tracing telemetry.
Arguments
name of a method, property or attribute on the charm type that returns an optional (fully resolvable) tempo url to which the charm traces will be pushed. If None, tracing will be effectively disabled.
name of a method, property or attribute on the charm type that returns an optional absolute path to a CA certificate file to be used when sending traces to a remote server. If it returns None, an insecure connection will be used. To avoid errors in transient situations where the endpoint is already https but there is no certificate on disk yet, it is recommended to disable tracing (by returning None from the tracing_endpoint) altogether until the cert has been written to disk.
service name tag to attach to all traces generated by this charm. Defaults to the juju application name this charm is deployed under.
pass any number of types that you also wish to autoinstrument. For example, charm libs, relation endpoint wrappers, workload abstractions, ...
Description
Use this function to get out-of-the-box traces for all events emitted on this charm and all method calls on instances of this class.
Usage:
from charms.tempo_k8s.v1.charm_tracing import trace_charm from charms.tempo_k8s.v1.tracing import TracingEndpointRequirer from ops import CharmBase
@trace_charm( tracing_endpoint="tempo_otlp_http_endpoint", ) class MyCharm(CharmBase):
def __init__(self, framework: Framework): ... self.tracing = TracingEndpointRequirer(self) @property def tempo_otlp_http_endpoint(self) -> Optional[str]: if self.tracing.is_ready(): return self.tracing.otlp_http_endpoint() else: return None
Methods
def trace_type(cls: _T)
Set up tracing on this class.
Description
Use this decorator to get out-of-the-box traces for all method calls on instances of this class.
It assumes that this class is only instantiated after a charm type decorated with @trace_charm
has been instantiated.
def
trace_method(
method: _F,
name
)
Trace this method.
Description
A span will be opened when this method is called and closed when it returns.
def
trace_function(
function: _F,
name
)
Trace this function.
Description
A span will be opened when this function is called and closed when it returns.
def trace(obj)
Trace this object and send the resulting spans to Tempo.
Description
It will dispatch to trace_type
if the decorated object is a class, otherwise
trace_function
.