How to deploy on AWS

Amazon Web Services is a popular subsidiary of Amazon that provides on-demand cloud computing platforms on a metered pay-as-you-go basis. Access the AWS web console at console.aws.amazon.com.

Summary

• Install AWS and Juju tooling
  • Authenticate
• Bootstrap Juju controller on AWS EC2
• Deploy charms
• Clean up

Install AWS and Juju tooling

Install Juju via snap:

sudo snap install juju

Follow the AWS documentation for guidance on how to install the Amazon Web Services CLI.

To check whether both Juju and AWS CLI are correctly installed, run commands to display their versions:

~$ juju version
3.5.4-genericlinux-amd64

~$ aws --version
aws-cli/2.13.25 Python/3.11.5 Linux/6.2.0-33-generic exe/x86_64.ubuntu.23 prompt/off

Authenticate

Create an IAM account or use legacy user access keys and secret key to operate AWS EC2:

mkdir -p ~/.aws && cat <<- EOF >  ~/.aws/credentials.yaml
credentials:
  aws:
    NAME_OF_YOUR_CREDENTIAL:
      auth-type: access-key
      access-key: SECRET_ACCESS_KEY_ID
      secret-key: SECRET_ACCESS_KEY_VALUE
EOF

Bootstrap Juju controller on AWS EC2

Add AWS credentials to Juju:

juju add-credential aws -f ~/.aws/credentials.yaml

Bootstrap Juju controller (check all supported configuration options):

juju bootstrap aws <CONTROLLER_NAME>

> juju bootstrap aws
Creating Juju controller "aws-us-east-1" on aws/us-east-1
Looking for packaged Juju agent version 3.5.4 for amd64
Located Juju agent version 3.5.4-ubuntu-amd64 at https://juju-dist-aws.s3.amazonaws.com/agents/agent/3.5.4/juju-3.5.4-linux-amd64.tgz
Launching controller instance(s) on aws/us-east-1...
 - i-0f4615983d113166d (arch=amd64 mem=8G cores=2)           
Installing Juju agent on bootstrap instance
Waiting for address
Attempting to connect to 54.226.221.6:22
Attempting to connect to 172.31.20.34:22
Connected to 54.226.221.6
Running machine configuration script...
Bootstrap agent now started
Contacting Juju controller at 54.226.221.6 to verify accessibility...

Bootstrap complete, controller "aws-us-east-1" is now available
Controller machines are in the "controller" model

Now you can run
	juju add-model <model-name>
to create a new model to deploy workloads.

Deploy charms

Create a new Juju model, if needed:

juju add-model <MODEL_NAME>

juju model-config logging-config='<root>=INFO;unit=DEBUG'

Then, Charmed Kafka can be deployed as usual. However, note that the smallest instance types on AWS may not have enough resources for hosting a Kafka broker. We recommend selecting an instance type that provides at the very least 8 GB of RAM and 4 cores. For more guidance on production environment sizing, see the Requirements page. You can find more information about the available instance types in the AWS documentation, ensuring that you select the correct AWS.

Deploy and integrate Kafka and ZooKeeper, for example:

juju deploy zookeeper -n3 --channel 3/stable
juju deploy kafka -n3 --constraints "instance-type=m7i.xlarge" --channel 3/stable
juju integrate kafka zookeeper

We also recommend to deploy a Data Integrator for creating an admin user to manage the content of the Kafka cluster:

juju deploy data-integrator admin \
  --config extra-user-roles=admin \
  --config topic-name=admin-topic

And integrate it with the Kafka application:

juju integrate kafka admin

For more information on Data Integrator and how to use it, please refer to the how-to manage applications guide.

Clean up

To list all controllers use the juju controllers command.

To destroy the Juju controller and remove AWS instance (Warning: all your data will be permanently deleted):

juju destroy-controller <CONTROLLER_NAME> --destroy-all-models --destroy-storage --force

Use juju list-controllers to list the available controllers the client has registered to.

After destroying the controller, check and manually delete all unnecessary AWS EC2 instances, to show the list of all your EC2 instances run the following command (make sure to use the correct region):

aws ec2 describe-instances --region us-east-1 --query "Reservations[].Instances[*].{InstanceType: InstanceType, InstanceId: InstanceId, State: State.Name}" --output table

-------------------------------------------------------
|                  DescribeInstances                  |
+---------------------+----------------+--------------+
|     InstanceId      | InstanceType   |    State     |
+---------------------+----------------+--------------+
|  i-0f374435695ffc54c|  m7i.xlarge    |  terminated  |
|  i-0e1e8279f6b2a08e0|  m7i.xlarge    |  terminated  |
|  i-061e0d10d36c8cffe|  m7i.xlarge    |  terminated  |
|  i-0f4615983d113166d|  m7i.xlarge    |  terminated  |
+---------------------+----------------+--------------+

List your Juju credentials with the juju credentials command:

...
Client Credentials:
Cloud        Credentials
aws          NAME_OF_YOUR_CREDENTIAL
...

Remove AWS EC2 CLI credentials from Juju:

juju remove-credential aws NAME_OF_YOUR_CREDENTIAL

Finally, remove AWS CLI user credentials (to avoid forgetting and leaking):

rm -f ~/.aws/credentials.yaml