Apache Kafka

Channel Revision Published Runs on
3/stable 185 23 Oct 2024
Ubuntu 22.04
3/candidate 195 Yesterday
Ubuntu 22.04
3/beta 194 03 Dec 2024
Ubuntu 22.04
3/edge 193 26 Nov 2024
Ubuntu 22.04
juju deploy kafka --channel 3/edge
Show information

Platform:

Ubuntu
22.04

Security Hardening Guide

This document provides guidance and instructions to achieve a secure deployment of Charmed Apache Kafka, including setting up and managing a secure environment. The document is divided into the following sections:

  1. Environment, outlining the recommendation for deploying a secure environment
  2. Applications, outlining the product features that enable a secure deployment of an Apache Kafka cluster
  3. Additional resources, providing any further information about security and compliance

Environment

The environment where applications operate can be divided in two components:

  1. Cloud
  2. Juju

Cloud

Charmed Apache Kafka can be deployed on top of several clouds and virtualization layers:

Juju

Juju is the component responsible for orchestrating the entire lifecycle, from deployment to Day 2 operations, of all applications. Therefore, it is imperative that it is set up securely. Please refer to the Juju documentation for more information on:

Cloud credentials

When configuring the cloud credentials to be used with Juju, ensure that the users have correct permissions to operate at the required level. Juju superusers responsible for bootstrapping and managing controllers require elevated permissions to manage several kinds of resources, such as virtual machines, networks, storages, etc. Please refer to the references below for more information on the policies required to be used depending on the cloud.

Juju users

It is very important that the different juju users are set up with minimal permission depending on the scope of their operations. Please refer to the User access levels documentation for more information on the access level and corresponding abilities that the different users can be granted.

Juju user credentials must be stored securely and rotated regularly to limit the chances of unauthorized access due to credentials leakage.

Applications

In the following, we provide guidance on how to harden your deployment using:

  1. Base images
  2. Charmed operator security upgrades
  3. Encryption
  4. Authentication
  5. Monitoring and auditing

Base images

Charmed Apache Kafka and Charmed Apache ZooKeeper currently run on top of Ubuntu 22.04. Deploy a Landscape Client Charm in order to connect the underlying VM to a Landscape User Account to manage security upgrades and integrate Ubuntu Pro subscriptions.

Charmed operator security upgrades

Charmed Apache Kafka and Charmed Apache ZooKeeper operators install a pinned revision of the Charmed Apache Kafka snap and Charmed ZooKeeper snap, respectively, to provide reproducible and secure environments. New versions of Charmed Apache Kafka and Charmed Apache ZooKeeper may be released to provide patching of vulnerabilities (CVEs). It is important to refresh the charm regularly to make sure the workload is as secure as possible. For more information on how to refresh the charm, see the how-to upgrade guide.

Encryption

Charmed Apache Kafka must be deployed with encryption enabled. To do that, you need to relate Apache Kafka and Apache ZooKeeper charms to one of the TLS certificate operator charms. Please refer to the Charming Security page for more information on how to select the right certificate provider for your use case.

For more information on encryption setup, see the How to enable encryption guide.

Authentication

Charmed Apache Kafka supports the following authentication layers:

  1. SCRAM-based SASL Authentication
  2. certificate-base Authentication (mTLS)
  3. OAuth Authentication using Hydra or Google

Each combination of authentication scheme and encryption is associated to the dedicated listener and it maps to a well-defined port. Please refer to the listener reference documentation for more information.

Monitoring and Auditing

Charmed Apache Kafka provides native integration with the Canonical Observability Stack (COS). To reduce the blast radius of infrastructure disruptions, the general recommendation is to deploy COS and the observed application into separate environments, isolated one another. Refer to the COS production deployments best practices for more information.

Refer to How-To user guide for more information on:

External user access to Apache Kafka is logged to the kafka-authorizer.log that is pushed to Loki endpoint and exposed via Grafana, both components being part of the COS stack. Access denials are logged at the INFO level, whereas allowed accesses are logged at the DEBUG level. Depending on the auditing needs, customize the logging level either for all logs via the log_level config option or only tune the logging level of the authorizerAppender in the log4j.properties file. Refer to the Reference documentation, for more information about the file system paths.

Additional Resources

For further information and details on the security and cryptographic specifications used by Charmed Apache Kafka, please refer to the Security Explanation page.