Apache Kafka
- Canonical
- Databases
Channel | Revision | Published | Runs on |
---|---|---|---|
3/stable | 185 | 23 Oct 2024 | |
3/candidate | 185 | 21 Oct 2024 | |
3/beta | 185 | 21 Oct 2024 | |
3/edge | 186 | 24 Oct 2024 |
juju deploy kafka --channel 3/stable
Deploy universal operators easily with Juju, the Universal Operator Lifecycle Manager.
Platform:
How to deploy Charmed Kafka
To deploy a Charmed Kafka cluster on a bare environment, it is necessary to:
- Set up a Juju Controller
- Set up a Juju Model
- Deploy Charmed Kafka and Charmed ZooKeeper
- (Optionally) Create an external admin user
In the next subsections, we will cover these steps separately by referring to relevant Juju documentation and providing details on the Charmed Kafka specifics. If you already have a Juju controller and/or a Juju model, you can skip the associated steps.
Juju controller setup
Before deploying Kafka, make sure you have a Juju controller accessible from your local environment using the Juju client snap.
The properties of your current controller can be listed using juju show-controller
.
Make sure that the controller’s back-end cloud is not K8s.
The cloud information can be retrieved with the following command
juju show-controller | yq '.[].details.cloud'
IMPORTANT If the cloud is
k8s
, please refer to the Charmed Kafka K8s documentation instead.
You can find more information on how to bootstrap and configure a controller for different
clouds here.
Make sure you bootstrap a machine
Juju controller.
Juju model setup
You can create a new Juju model using
juju add-model <model>
Alternatively, you can use a pre-existing Juju model and switch to it by running the following command:
juju switch <model-name>
Make sure that the model is not a k8s
type. The type of the model
can be obtained by
juju show-model | yq '.[].type'
IMPORTANT If the model is
k8s
, please refer to the Charmed Kafka K8s documentation instead.
Deploy Charmed Kafka and Charmed ZooKeeper
The Kafka and ZooKeeper charms can both be deployed as follows:
$ juju deploy kafka --channel 3/stable -n <kafka-units> --trust
$ juju deploy zookeeper --channel 3/stable -n <zookeeper-units>
where <kafka-units>
and <zookeeper-units>
– the number of units to deploy for Kafka and ZooKeeper. We recommend values of at least 3
and 5
respectively.
NOTE The
--trust
option is needed for the Kafka application if NodePort is used. For more information about the trust options usage, see the Juju documentation.
After this, it is necessary to connect them:
$ juju relate kafka zookeeper
Once all the units show as active|idle
in the juju status
output, the deployment
should be ready to be used.
(Optional) Create an external admin users
Charmed Kafka aims to follow the secure by default paradigm. As a consequence, after being deployed the Kafka cluster won’t expose any external listener. In fact, ports are only opened when client applications are related, also depending on the protocols to be used. Please refer to this table for more information about the available listeners and protocols.
It is however generally useful for most of the use-cases to create a first admin user to be used to manage the Kafka cluster (either internally or externally).
To create an admin user, deploy the Data Integrator Charm with
extra-user-roles
set to admin
juju deploy data-integrator --channel stable --config topic-name=test-topic --config extra-user-roles=admin
and relate to the Kafka charm
juju relate data-integrator kafka
To retrieve authentication information such as the username, password, etc. use
juju run data-integrator/leader get-credentials