Kafka
- By Canonical Data Platform
Channel | Revision | Published | Runs on |
---|---|---|---|
3/stable | 123 | 24 May 2023 | |
3/edge | 123 | 23 May 2023 |
juju deploy kafka --channel 3/stable
You will need Juju 2.9 to be able to run this command. Learn how to upgrade to Juju 2.9.
Deploy universal operators easily with Juju, the Universal Operator Lifecycle Manager.
Platform:
How to deploy and manage units
Basic usage
The Kafka and ZooKeeper operators can both be deployed as follows:
$ juju deploy zookeeper --channel 3/edge -n 5
$ juju deploy kafka --channel 3/edge -n 3
After this, it is necessary to connect them:
$ juju relate kafka zookeeper
To watch the process, juju status
can be used. Once all the units show as active|idle
the credentials to access a broker can be queried with:
juju run-action kafka/leader get-admin-credentials --wait
Apache Kafka ships with bin/*.sh
commands to do various administrative tasks, e.g bin/kafka-config.sh
to update cluster configuration, bin/kafka-topics.sh
for topic management, and many more! The Kafka Charmed Operator provides these commands to administrators to easily run their desired cluster configurations securely with SASL authentication, either from within the cluster or as an external client.
If you wish to run a command from the cluster, in order to (for example) list the current topics on the Kafka cluster, you can run:
BOOTSTRAP_SERVERS=$(juju run-action kafka/leader get-admin-credentials --wait | grep "bootstrap.servers" | cut -d "=" -f 2)
juju ssh kafka/leader 'charmed-kafka.topics --bootstrap-server $BOOTSTRAP_SERVERS --list --command-config /var/snap/charmed-kafka/common/client.properties'
Note that when no other application is related to Kafka, the cluster is secured-by-default and listeners are disabled, thus preventing any incoming connection. However, even for running the commands above, listeners must be enable. If there is no other application, deploy a data-integrator
charm and relate it to Kafka, as outlined in the Relation section to enable listeners.
Available Kafka bin commands can be found with:
snap info charmed-kafka --channel 3/edge
Replication
Scaling up
The charm can be scaled up using juju add-unit
command.
juju add-unit kafka
To add a specific number of brokers, an extra argument is needed:
juju add-unit kafka -n <num_brokers_to_add>
Scaling down
To scale down the charm, use juju remove-unit
command.
juju remove-unit <unit_name>
Even when scaling multiple units at the same time, the charm uses a rolling restart sequence to make sure the cluster stays available and healthy during the operation.
Help us improve this documentation
Most of this documentation can be collaboratively discussed and changed on the respective topic in the doc category of the Charmhub forum. See the documentation guidelines if you’d like to contribute.