Tempo

  • By Canonical Observability
Channel Revision Published Runs on
latest/stable 30 24 Apr 2024
Ubuntu 22.04
latest/candidate 33 24 Apr 2024
Ubuntu 22.04
latest/beta 36 24 Apr 2024
Ubuntu 22.04
latest/edge 46 08 May 2024
Ubuntu 22.04
juju deploy tempo-k8s
Show information

Learn to deploy on juju >

Platform:

charms.tempo_k8s.v0.tempo_scrape

Overview.

This document explains how to integrate with the Tempo charm for the purpose of providing a tracing endpoint to Tempo. It also explains how alternative implementations of the Tempo charm may maintain the same interface and be backward compatible with all currently integrated charms.

Provider Library Usage

This Tempo charm interacts with its scrape targets using its charm library. Charms seeking to expose tracing endpoints for the Tempo charm, must do so using the TracingEndpointProvider object from this charm library. For the simplest use cases, using the TracingEndpointProvider object only requires instantiating it, typically in the constructor of your charm. The TracingEndpointProvider constructor requires the name of the relation over which a scrape target (tracing endpoint) is exposed to the Tempo charm. This relation must use the tempo_scrape interface. By default address of the tracing endpoint is set to the unit IP address, by each unit of the TracingEndpointProvider charm. These units set their address in response to the PebbleReady event of each container in the unit, since container restarts of Kubernetes charms can result in change of IP addresses. The default name for the tracing endpoint relation is tracing. It is strongly recommended to use the same relation name for consistency across charms and doing so obviates the need for an additional constructor argument. The TracingEndpointProvider object may be instantiated as follows

from charms.tempo_k8s.v0.tempo_scrape import TracingEndpointProvider

def __init__(self, *args):
    super().__init__(*args)
    # ...
    self.tracing_endpoint = TracingEndpointProvider(self)
    # ...

Note that the first argument (self) to TracingEndpointProvider is always a reference to the parent (scrape target) charm.

An instantiated TracingEndpointProvider object will ensure that each unit of its parent charm, is a scrape target for the TracingEndpointConsumer (Tempo) charm. By default TracingEndpointProvider assumes each unit of the consumer charm exports its profiles on port 80. These defaults may be changed by providing the TracingEndpointProvider constructor an optional argument (jobs) that represents a Tempo scrape job specification using Python standard data structures. This job specification is a subset of Tempo's own scrape configuration format but represented using Python data structures. More than one job may be provided using the jobs argument. Hence jobs accepts a list of dictionaries where each dictionary represents one <scrape_config> object as described in the Tempo documentation. The currently supported configuration subset is: job_name, static_configs

Suppose it is required to change the port on which scraped profiles are exposed to 8000. This may be done by providing the following data structure as the value of jobs.

[{"static_configs": [{"targets": ["*:8000"]}]}]

The wildcard ("*") host specification implies that the scrape targets will automatically be set to the host addresses advertised by each unit of the consumer charm.

It is also possible to change the profile path and scrape multiple ports, for example

[{"static_configs": [{"targets": ["*:8000", "*:8081"]}]}]

More complex scrape configurations are possible. For example

[{
    "static_configs": [{
        "targets": ["10.1.32.215:7000", "*:8000"],
        "labels": {
            "some-key": "some-value"
        }
    }]
}]

This example scrapes the target "10.1.32.215" at port 7000 in addition to scraping each unit at port 8000. There is however one difference between wildcard targets (specified using "*") and fully qualified targets (such as "10.1.32.215"). The Tempo charm automatically associates labels with profiles generated by each target. These labels localise the source of profiles within the Juju topology by specifying its "model name", "model UUID", "application name" and "unit name". However unit name is associated only with wildcard targets but not with fully qualified targets.

Multiple jobs with labels are allowed, but each job must be given a unique name:

[
    {
        "job_name": "my-first-job",
        "static_configs": [
            {
                "targets": ["*:7000"],
                "labels": {
                    "some-key": "some-value"
                }
            }
        ]
    },
    {
        "job_name": "my-second-job",
        "static_configs": [
            {
                "targets": ["*:8000"],
                "labels": {
                    "some-other-key": "some-other-value"
                }
            }
        ]
    }
]

Important: job_name should be a fixed string (e.g. hardcoded literal). For instance, if you include variable elements, like your unit.name, it may break the continuity of the profile time series gathered by Tempo when the leader unit changes (e.g. on upgrade or rescale).

Consumer Library Usage

The TracingEndpointConsumer object may be used by Tempo charms to manage relations with their scrape targets. For this purposes a Tempo charm needs to do two things

  1. Instantiate the TracingEndpointConsumer object by providing it a reference to the parent (Tempo) charm and optionally the name of the relation that the Tempo charm uses to interact with scrape targets. This relation must confirm to the tempo_scrape interface and it is strongly recommended that this relation be named tracing which is its default value.

For example a Tempo charm may instantiate the TracingEndpointConsumer in its constructor as follows

from charms.tempo_k8s.v0.tempo_scrape import TracingEndpointConsumer

def __init__(self, *args):
    super().__init__(*args)
    # ...
    self.tracing_consumer = TracingEndpointConsumer(self)
    # ...

  1. A Tempo charm also needs to respond to the TargetsChangedEvent event of the TracingEndpointConsumer by adding itself as an observer for these events, as in

    self.framework.observe( self.tracing_consumer.on.targets_changed, self._on_scrape_targets_changed, )

In responding to the TargetsChangedEvent event the Tempo charm must update the Tempo configuration so that any new scrape targets are added and/or old ones removed from the list of scraped endpoints. For this purpose the TracingEndpointConsumer object exposes a jobs() method that returns a list of scrape jobs. Each element of this list is the Tempo scrape configuration for that job. In order to update the Tempo configuration, the Tempo charm needs to replace the current list of jobs with the list provided by jobs() as follows

def _on_scrape_targets_changed(self, event):
    ...
    scrape_jobs = self.tracing_consumer.jobs()
    for job in scrape_jobs:
        tempo_scrape_config.append(job)
    ...
Relation Data

Units of profiles provider charms advertise their names and addresses over unit relation data using the tempo_scrape_unit_name and tempo_scrape_unit_address keys. While the scrape_metadata, scrape_jobs and alert_rules keys in application relation data of profiles provider charms hold eponymous information.

Index

class RelationNotFoundError

Description

Raised if no relation with the given name is found. None

Methods

RelationNotFoundError. __init__( self , relation_name: str )

class RelationInterfaceMismatchError

Description

Raised if the relation with the given name has an unexpected interface. None

Methods

RelationInterfaceMismatchError. __init__( self , relation_name: str , expected_relation_interface: str , actual_relation_interface: str )

class RelationRoleMismatchError

Description

Raised if the relation with the given name has a different role than expected. None

Methods

RelationRoleMismatchError. __init__( self , relation_name: str , expected_relation_role: RelationRole , actual_relation_role: RelationRole )

class TargetsChangedEvent

Description

Event emitted when Tempo scrape targets change. None

Methods

TargetsChangedEvent. __init__( self , handle , relation_id )

TargetsChangedEvent. snapshot( self )

Description

Save scrape target relation information. None

TargetsChangedEvent. restore( self , snapshot )

Description

Restore scrape target relation information. None

class MonitoringEvents

Description

Event descriptor for events raised by TracingEndpointConsumer. None

class TracingEndpointRequirer

Description

A Tempo based monitoring service. None

Methods

TracingEndpointRequirer. __init__( self , charm: CharmBase , tempo_endpoint , relation_name: str )

A Tempo based Monitoring service.

Arguments

charm

a CharmBase instance that manages this instance of the Tempo service.

relation_name

an optional string name of the relation between charm and the Tempo charmed service. The default is "tracing".

TracingEndpointRequirer. update_relation_data( self , _ )

class EndpointChangedEvent

class TracingEndpointEvents

class TracingEndpointProvider

Description

A tracing endpoint for Tempo. None

Methods

TracingEndpointProvider. __init__( self , charm: CharmBase , relation_name: str )

Construct a tracing provider for a Tempo charm.

Arguments

charm

a CharmBase object that manages this TracingEndpointProvider object. Typically this is self in the instantiating class.

relation_name

an optional string name of the relation between charm and the Tempo charmed service. The default is "tracing". It is strongly advised not to change the default, so that people deploying your charm will have a consistent experience with all other charms that provide tracing endpoints.

Description

If your charm exposes a Tempo tracing endpoint, the TracingEndpointProvider object enables your charm to easily communicate how to reach that endpoint.

By default, a charm instantiating this object has the tracing endpoints of each of its units scraped by the related Tempo charms.

The scraped profiles are automatically tagged by the Tempo charms with Juju topology data via the juju_model_name, juju_model_uuid, juju_application_name and juju_unit labels. To support such tagging TracingEndpointProvider automatically forwards scrape metadata to a TracingEndpointConsumer (Tempo charm).

Scrape targets provided by TracingEndpointProvider can be customized when instantiating this object. For example in the case of a charm exposing the tracing endpoint for each of its units on port 8080, the TracingEndpointProvider can be instantiated as follows:

self.tracing_endpoint_provider = TracingEndpointProvider(
    self, jobs=[{"static_configs": [{"targets": ["*:8080"]}]}]
)

The notation *:<port> means "scrape each unit of this charm on port <port>.

TracingEndpointProvider. endpoint( self )

TracingEndpointProvider. otlp_grpc_endpoint( self )