Kafka Test App

The test application charm used in various test scenarios for Charmed Kafka K8s/VM installations.

While this tutorial intends to guide and teach you as you deploy Kafka Test App, it will be most beneficial if you already have a familiarity with:

• Basic terminal commands.
• Kafka concepts such as topics, producers and consumers.

Kafka Test App is the workload-less charm. It can be deployed and used in both K8s and VM installations.

Just deploy and relate it:

# Deploy
juju deploy kafka-test-app producer --config role=producer --channel edge
juju deploy kafka-test-app consumer --config role=consumer --channel edge
juju deploy kafka
juju deploy zookeeper

# Integrate
juju integrate kafka producer
juju integrate kafka consumer
juju integrate kafka zookeeper

# The endless test will start automatically, follow:
juju ssh consumer/leader tail -f /tmp/*_consumer.log

==> /tmp/1714675860_consumer.log <==
2024-05-02 18:58:09,873 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=4, offset=145, timestamp=1714676289867, timestamp_type=0, key=None, value=b'{"timestamp": 1714676289.866915, "_id": "b09368e996e0463a81859f09172ce402", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #821"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)
2024-05-02 18:58:10,375 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=4, offset=146, timestamp=1714676290371, timestamp_type=0, key=None, value=b'{"timestamp": 1714676290.371619, "_id": "fde91ab429bd40f1b07f765498ebdff5", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #822"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)
2024-05-02 18:58:10,882 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=2, offset=177, timestamp=1714676290875, timestamp_type=0, key=None, value=b'{"timestamp": 1714676290.8754, "_id": "1208f40acb5441588e036a654aff43af", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #823"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=143, serialized_header_size=-1)
2024-05-02 18:58:11,382 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=2, offset=178, timestamp=1714676291380, timestamp_type=0, key=None, value=b'{"timestamp": 1714676291.380164, "_id": "7e3528a19d15473ca0e3d6d9ab9d09aa", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #824"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)
2024-05-02 18:58:11,888 INFO [__main__] (MainThread) (<module>) ConsumerRecord(topic='test-topic', partition=2, offset=179, timestamp=1714676291882, timestamp_type=0, key=None, value=b'{"timestamp": 1714676291.882081, "_id": "1ff770274d224cb38c7a758e243a38cc", "origin": "juju-b12b18-3 (10.184.219.11)", "content": "Message #825"}', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=145, serialized_header_size=-1)

Example of juju status output:

Model   Controller  Cloud/Region         Version  SLA          Timestamp
test53  lxd         localhost/localhost  3.4.2    unsupported  20:51:50+02:00

App        Version  Status  Scale  Charm           Channel      Rev  Exposed  Message
consumer            active      1  kafka-test-app  latest/edge   10  no       Topic test-topic enabled with process consumer
kafka               active      1  kafka           3/stable     156  no       
producer            active      1  kafka-test-app  latest/edge   10  no       Topic test-topic enabled with process producer
zookeeper           active      1  zookeeper       3/stable     126  no       

Unit          Workload  Agent  Machine  Public address  Ports  Message
consumer/0*   active    idle   2        10.184.219.5           Topic test-topic enabled with process consumer
kafka/0*      active    idle   0        10.184.219.254         
producer/0*   active    idle   3        10.184.219.11          Topic test-topic enabled with process producer
zookeeper/0*  active    idle   1        10.184.219.31          

Machine  State    Address         Inst id        Base          AZ  Message
0        started  10.184.219.254  juju-b12b18-0  ubuntu@22.04      Running
1        started  10.184.219.31   juju-b12b18-1  ubuntu@22.04      Running
2        started  10.184.219.5    juju-b12b18-2  ubuntu@22.04      Running
3        started  10.184.219.11   juju-b12b18-3  ubuntu@22.04      Running

Integration provider  Requirer                Interface     Type     Message
consumer:app-peers    consumer:app-peers      app-peers     peer     
kafka:cluster         kafka:cluster           cluster       peer     
kafka:kafka-client    consumer:kafka-cluster  kafka_client  regular  
kafka:kafka-client    producer:kafka-cluster  kafka_client  regular  
kafka:restart         kafka:restart           rolling_op    peer     
kafka:upgrade         kafka:upgrade           upgrade       peer     
producer:app-peers    producer:app-peers      app-peers     peer     
zookeeper:cluster     zookeeper:cluster       cluster       peer     
zookeeper:restart     zookeeper:restart       rolling_op    peer     
zookeeper:upgrade     zookeeper:upgrade       upgrade       peer     
zookeeper:zookeeper   kafka:zookeeper         zookeeper     regular