Observability Libs

  • Jon Seager
Channel Revision Published Runs on
latest/edge 50 11 Oct 2024
Ubuntu 20.04
juju deploy observability-libs --channel edge
Show information

Learn to deploy on juju >

Platform:

charms.observability_libs.v0.kubernetes_compute_resources_patch

KubernetesComputeResourcesPatch Library.

This library is designed to enable developers to more simply patch the Kubernetes compute resource limits and requests created by Juju during the deployment of a charm.

When initialised, this library binds a handler to the parent charm's config-changed event. The config-changed event is used because it is guaranteed to fire on startup, on upgrade and on pod churn. Additionally, resource limits may be set by charm config options, which would also be caught out-of-the-box by this handler. The handler applies the patch to the app's StatefulSet. This should ensure that the resource limits are correct throughout the charm's life. Additional optional user-provided events for re-applying the patch are supported but discouraged.

The constructor takes a reference to the parent charm, a 'limits' and a 'requests' dictionaries that together define the resource requirements. For information regarding the lightkube ResourceRequirements model, please visit the lightkube docs.

Getting Started

To get started using the library, you just need to fetch the library using charmcraft. Note that you also need to add lightkube and lightkube-models to your charm's requirements.txt.

cd some-charm
charmcraft fetch-lib charms.observability_libs.v0.kubernetes_compute_resources_patch
cat << EOF >> requirements.txt
lightkube
lightkube-models
EOF

Then, to initialise the library:

# ...
from charms.observability_libs.v0.kubernetes_compute_resources_patch import (
    KubernetesComputeResourcesPatch,
    ResourceRequirements,
)

class SomeCharm(CharmBase):
  def __init__(self, *args):
    # ...
    self.resources_patch = KubernetesComputeResourcesPatch(
        self,
        "container-name",
        resource_reqs_func=lambda: ResourceRequirements(
            limits={"cpu": "2"}, requests={"cpu": "1"}
        ),
    )
    self.framework.observe(self.resources_patch.on.patch_failed, self._on_resource_patch_failed)

  def _on_resource_patch_failed(self, event):
    self.unit.status = BlockedStatus(event.message)
    # ...

Or, if, for example, the resource specs are coming from config options:

class SomeCharm(CharmBase):
  def __init__(self, *args):
    # ...
    self.resources_patch = KubernetesComputeResourcesPatch(
        self,
        "container-name",
        resource_reqs_func=self._resource_spec_from_config,
    )

  def _resource_spec_from_config(self) -> ResourceRequirements:
    spec = {"cpu": self.model.config.get("cpu"), "memory": self.model.config.get("memory")}
    return ResourceRequirements(limits=spec, requests=spec)

If you wish to pull the state of the resources patch operation and set the charm unit status based on that patch result, you can achieve that using get_status() function.

class SomeCharm(CharmBase):
    def __init__(self, *args):
        #...
        self.framework.observe(self.on.collect_unit_status, self._on_collect_unit_status)
    #...
    def _on_collect_unit_status(self, event: CollectStatusEvent):
        event.add_status(self.resources_patch.get_status())

Additionally, you may wish to use mocks in your charm's unit testing to ensure that the library does not try to make any API calls, or open any files during testing that are unlikely to be present, and could break your tests. The easiest way to do this is during your test setUp:

# ...
from ops import ActiveStatus

@patch.multiple(
    "charm.KubernetesComputeResourcesPatch",
    _namespace="test-namespace",
    _is_patched=lambda *a, **kw: True,
    is_ready=lambda *a, **kw: True,
    get_status=lambda _: ActiveStatus(),
)
@patch("lightkube.core.client.GenericSyncClient")
def setUp(self, *unused):
    self.harness = Harness(SomeCharm)
    # ...

References:

  • https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
  • https://gtsystem.github.io/lightkube-models/1.23/models/core_v1/#resourcerequirements

Index

def adjust_resource_requirements(
    limits,
    requests,
    adhere_to_requests: bool
)

Adjust resource limits so that limits and requests are consistent with each other.

Arguments

limits

the "limits" portion of the resource spec.

requests

the "requests" portion of the resource spec.

adhere_to_requests

a flag indicating which portion should be adjusted when "limits" is lower than "requests":

  • if True, "limits" will be adjusted to max(limits, requests).
  • if False, "requests" will be adjusted to min(limits, requests).

Returns

An adjusted (limits, requests) 2-tuple.

def is_valid_spec(
    spec,
    debug
)

Check if the spec dict is valid.

Description

TODO: generally, the keys can be anything, not just cpu and memory. Perhaps user could pass list of custom allowed keys in addition to the K8s ones?

def sanitize_resource_spec_dict(spec)

Fix spec values without altering semantics.

Description

The purpose of this helper function is to correct known issues. This function is not intended for fixing user mistakes such as incorrect keys present; that is left for the is_valid_spec function.

class K8sResourcePatchFailedEvent

Description

Emitted when patching fails. None

Methods

K8sResourcePatchFailedEvent. __init__( self , handle , message )

K8sResourcePatchFailedEvent. snapshot( self )

Description

Save grafana source information. None

K8sResourcePatchFailedEvent. restore( self , snapshot )

Description

Restore grafana source information. None

class K8sResourcePatchEvents

Description

Events raised by :class:K8sResourcePatchEvents. None

class ContainerNotFoundError

Description

Raised when a given container does not exist in the list of containers. None

class ResourcePatcher

Description

Helper class for patching a container's resource limits in a given StatefulSet. None

Methods

ResourcePatcher. __init__( self , namespace: str , statefulset_name: str , container_name: str )

ResourcePatcher. is_patched( self , resource_reqs: ResourceRequirements )

Reports if the resource patch has been applied to the StatefulSet.

Returns

bool

A boolean indicating if the service patch has been applied.

ResourcePatcher. get_templated( self )

Description

Returns the resource limits specified in the StatefulSet template. None

ResourcePatcher. get_actual( self , pod_name: str )

Description

Return the resource limits that are in effect for the container in the given pod. None

ResourcePatcher. is_failed( self , resource_reqs_func )

Returns a tuple indicating whether a patch operation has failed along with a failure message.

Description

Implementation is based on dry running the patch operation to catch if there would be failures (e.g: Wrong spec and Auth errors).

ResourcePatcher. is_in_progress( self )

Returns a boolean to indicate whether a patch operation is in progress.

Description

Implementation follows a similar approach to kubectl rollout status statefulset to track the progress of a rollout. Reference: https://github.com/kubernetes/kubectl/blob/kubernetes-1.31.0/pkg/polymorphichelpers/rollout_status.go

ResourcePatcher. is_ready( self , pod_name , resource_reqs: ResourceRequirements )

Reports if the resource patch has been applied and is in effect.

Returns

bool

A boolean indicating if the service patch has been applied and is in effect.

ResourcePatcher. apply( self , resource_reqs: ResourceRequirements , dry_run )

Description

Patch the Kubernetes resources created by Juju to limit cpu or mem. None

class KubernetesComputeResourcesPatch

Description

A utility for patching the Kubernetes compute resources set up by Juju. None

Methods

KubernetesComputeResourcesPatch. __init__( self , charm: CharmBase , container_name: str )

Constructor for KubernetesComputeResourcesPatch.

Arguments

charm

the charm that is instantiating the library.

container_name

the container for which to apply the resource limits.

resource_reqs_func

a callable returning a ResourceRequirements; if raises, should only raise ValueError.

refresh_event

an optional bound event or list of bound events which will be observed to re-apply the patch.

Description

References: - https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

KubernetesComputeResourcesPatch. is_ready( self )

Reports if the resource patch has been applied and is in effect.

Returns

bool

A boolean indicating if the service patch has been applied and is in effect.

KubernetesComputeResourcesPatch. get_status( self )

Return the status of patching the resource limits in a StatusBase format.

Returns

StatusBase

There is a 1:1 mapping between the state of the patching operation and a StatusBase value that the charm can be set to.