Kafka
- By Canonical Data Platform
| Channel | Revision | Published | Runs on |
|---|---|---|---|
| 3/stable | 123 | 24 May 2023 | |
| 3/edge | 123 | 23 May 2023 |
juju deploy kafka --channel 3/stable
You will need Juju 2.9 to be able to run this command. Learn how to upgrade to Juju 2.9.
Deploy universal operators easily with Juju, the Universal Operator Lifecycle Manager.
Platform:
Relate your Kafka deployment
This is part of the Charmed Kafka Tutorial. Please refer to this page for more information and the overview of the content.
Relations
Relations, or what Juju documentation describes as Integration, are the easiest way to create a user for Kafka in Charmed Kafka. Relations automatically create a username, password, and topic for the desired user/application.
Data Integrator Charm
To start using the Kafka cluster, we will now relate our application to the Data Integrator Charm. This is a bare-bones charm that allows for central management of database users, providing support for different kinds of data platforms (e.g. MongoDB, MySQL, PostgreSQL, Kafka, OpenSearch, etc) with a consistent, opinionated and robust user experience. In order to deploy the Data Integrator Charm we can use the command juju deploy we have learned above:
juju deploy data-integrator --channel stable --config topic-name=test-topic --config extra-user-roles=producer,consumer
The expected output:
Located charm "data-integrator" in charm-hub, revision 11
Deploying "data-integrator" from charm-hub charm "data-integrator", revision 11 in channel stable on jammy
Relate to Kafka
Now that the Database Integrator Charm has been set up, we can relate it to Kafka. This will automatically create a username, password, and database for the Database Integrator Charm. Relate the two applications with:
juju relate data-integrator kafka
Wait for juju status --watch 1s to show:
Model Controller Cloud/Region Version SLA Timestamp
tutorial overlord localhost/localhost 2.9.38 unsupported 10:04:50Z
App Version Status Scale Charm Channel Rev Exposed Message
data-integrator active 1 data-integrator stable 11 no
kafka active 3 kafka 3/stable 117 no
zookeeper active 5 zookeeper 3/stable 99 no
Unit Workload Agent Machine Public address Ports Message
data-integrator/0* active idle 8 10.244.26.4
kafka/0 active idle 5 10.244.26.43 machine system settings are not optimal - see logs for info
kafka/1* active idle 6 10.244.26.6 machine system settings are not optimal - see logs for info
kafka/2 active idle 7 10.244.26.19 machine system settings are not optimal - see logs for info
zookeeper/0 active idle 0 10.244.26.251
zookeeper/1 active idle 1 10.244.26.129
zookeeper/2 active idle 2 10.244.26.121
zookeeper/3* active idle 3 10.244.26.28
zookeeper/4 active idle 4 10.244.26.174
Machine State Address Inst id Series AZ Message
0 started 10.244.26.251 juju-f1a2cd-0 jammy Running
1 started 10.244.26.129 juju-f1a2cd-1 jammy Running
2 started 10.244.26.121 juju-f1a2cd-2 jammy Running
3 started 10.244.26.28 juju-f1a2cd-3 jammy Running
4 started 10.244.26.174 juju-f1a2cd-4 jammy Running
5 started 10.244.26.43 juju-f1a2cd-5 jammy Running
6 started 10.244.26.6 juju-f1a2cd-6 jammy Running
7 started 10.244.26.19 juju-f1a2cd-7 jammy Running
8 started 10.244.26.4 juju-f1a2cd-8 jammy Running
To retrieve information such as the username, password, and topic. Enter:
juju run-action data-integrator/leader get-credentials --wait
This should output something like:
unit-data-integrator-0:
UnitId: data-integrator/0
id: "4"
results:
kafka:
consumer-group-prefix: relation-6-
endpoints: 10.244.26.43:9092,10.244.26.6:9092,10.244.26.19:9092
password: ILg8C5msYRvqOnGATeFPyw2DKHncritf
tls: disabled
topic: test-topic
username: relation-6
zookeeper-uris: 10.244.26.121:2181,10.244.26.129:2181,10.244.26.174:2181,10.244.26.251:2181,10.244.26.28:2181/kafka
ok: "True"
status: completed
timing:
completed: 2023-04-25 10:09:26 +0000 UTC
enqueued: 2023-04-25 10:09:20 +0000 UTC
started: 2023-04-25 10:09:21 +0000 UTC
Save the value listed under bootstrap-server, username and password. (Note: your hostnames, usernames, and passwords will likely be different.)
Remove the user
To remove the user, remove the relation. Removing the relation automatically removes the user that was created when the relation was created. Enter the following to remove the relation:
juju remove-relation kafka data-integrator
The output of the juju model should be something like this:
Model Controller Cloud/Region Version SLA Timestamp
tutorial overlord localhost/localhost 2.9.38 unsupported 10:20:59Z
App Version Status Scale Charm Channel Rev Exposed Message
data-integrator blocked 1 data-integrator stable 11 no Please relate the data-integrator with the desired product
kafka active 3 kafka 3/stable 117 no
zookeeper active 5 zookeeper 3/stable 99 no
Unit Workload Agent Machine Public address Ports Message
data-integrator/0* blocked idle 8 10.244.26.4 Please relate the data-integrator with the desired product
kafka/0 active idle 5 10.244.26.43
kafka/1* active idle 6 10.244.26.6
kafka/2 active idle 7 10.244.26.19
zookeeper/0 active idle 0 10.244.26.251
zookeeper/1 active idle 1 10.244.26.129
zookeeper/2 active idle 2 10.244.26.121
zookeeper/3* active idle 3 10.244.26.28
zookeeper/4 active idle 4 10.244.26.174
Machine State Address Inst id Series AZ Message
0 started 10.244.26.251 juju-f1a2cd-0 jammy Running
1 started 10.244.26.129 juju-f1a2cd-1 jammy Running
2 started 10.244.26.121 juju-f1a2cd-2 jammy Running
3 started 10.244.26.28 juju-f1a2cd-3 jammy Running
4 started 10.244.26.174 juju-f1a2cd-4 jammy Running
5 started 10.244.26.43 juju-f1a2cd-5 jammy Running
6 started 10.244.26.6 juju-f1a2cd-6 jammy Running
7 started 10.244.26.19 juju-f1a2cd-7 jammy Running
8 started 10.244.26.4 juju-f1a2cd-8 jammy Running
Help us improve this documentation
Most of this documentation can be collaboratively discussed and changed on the respective topic in the doc category of the Charmhub forum. See the documentation guidelines if you’d like to contribute.