Grafana Agent

The grafana-agent charm has two limitations:

1. Only one instance of the charm should be related to each application.
2. Only one instance should run per machine.

Failing to enforce these constraints can lead to configuration conflicts and errors.

When applications allow multiple relations with different instances of grafana-agent (e.g., ga-one and ga-two), or several instances of grafana-agent are running in the same machine, several issues can arise:

1. Single service and config file: Even if multiple grafana-agent charms are related, only one service and one configuration file are installed on the machine, overwritten by all related charms.

2. Conflicting confinement settings: If different grafana-agent instances use different confinement types (classic vs strict), the snap will apply only one type, potentially causing inconsistencies.

3. Errors when removing relations: Removing a relation may uninstall the grafana-agent snap entirely, leaving other related charms in an error state as they attempt to manage a service that no longer exists.

More information about this, can be found in this post.

Recommended Implementation and Safeguards

• Applications should use a limit in their metadata.yaml / charmcraft.yaml to restrict relations to only one instance of grafana-agent.

    cos-agent:
      interface: cos_agent
      limit: 1
  

• Ensure that only one grafana-agent instance runs per machine to avoid configuration conflicts and unexpected behaviour.

• Two probes (in draft state right now) can be executed in order to verify whether only one grafana-agent is related to a charm or only one grafana-agent deployed in a machine:

  juju export-bundle | ./probe_bundle.py
  juju status --format=yaml | ./probe_status.py
  

Following these practices ensures stable and predictable deployments in production environments.