Spiculecharms Druid Config

Channel Version Revision Published Runs on
latest/stable 65 65 19 Mar 2021
Ubuntu 18.04 Ubuntu 16.04
latest/beta 59 59 19 Mar 2021
Ubuntu 16.04
latest/edge 62 62 19 Mar 2021
Ubuntu 18.04 Ubuntu 16.04
juju deploy spiculecharms-druid-config
Show information
You will need Juju 2.9 to be able to run this command. Learn how to upgrade to Juju 2.9.

Platform:

Ubuntu
16.04 18.04

About

Druid Configuration Charm by spiculecharms Read more


Relevant links


Discuss this charm

Share your thoughts on this charm with the community on discourse.

Join the discussion

Overview

Welcome to the Druid Config charm by Spicule. This charm is responsible for providing common configuration options to a Druid Cluster. It is a lightweight charm that provides configuration files to each node in your cluster. For this reason, it is not necessary to scale out this charm as you scale out your cluster. There should only be one unit of this charm per cluster, and all Druid nodes (Druid Overlord, Druid Coordinator, Druid Middlemanager, Druid Historical and Druid Broker) must be directly related to this charm.

Usage

To deploy this charm from the command line, enter the following:

juju deploy cs:~spiculecharms/druid-config --series xenial

That's it! This charm requires only the smallest instance of whatever Cloud platform you are using, and by applying no hardware constraints, Juju will automatically request the smallest instance available.

Once you have deployed your Druid Config charm, you must add a relation to each of your Druid nodes. This is simple to do from the command line:

juju add-relation druid-config druid-<druid-node-name>

MySQL and Zookeeper

In addition to relating Config to your Druid nodes, you must also relate it to a MySQL charm for metadata storage, as well as a Zookeeper charm for cluster coordination. Deploy these charms and add relations to them, and Config will automatically configure your cluster to point to these servers.

Storage Options

Druid Config is responsible for configuring your Druid Cluster's storage settings. How you set this up depends on your choice of storage solution. Druid currently supports three main storage options:

  • Local storage
  • S3 Buckets
  • HDFS

Local storage is enabled by default on deployment. To utilise local storage, simply spin up your Druid Config charm and relate it to your Druid nodes.

To configure your cluster to utilise an S3 bucket from the command line on deployment, enter the following:

juju deploy cs:~spiculecharms/druid-config --series xenial --config s3-storage="True" --config s3-bucket="<s3-bucket>" --config s3-accessKey="<s3-accessKey>" --config s3-secretKey="<s3-secretKey>"

To configure your cluster to utilise HDFS, simply add a relation to the Hadoop Plugin charm, which you can find in our handy Anssr Hadoop Bundle, and Druid Config will handle the rest for you!

Druid Config is also responsive to changes in these configuration settings, allowing the user to change their storage solution as needed. If you spin up a cluster using local storage at first, you can modify the configuration of Config to use S3 instead, and the Druid Nodes will re-configure and restart automatically. Likewise, if you wish to change to HDFS, simply spin up your Anssr Hadoop Bundle, relate the Hadoop Plugin to Config and your cluster will be re-configured to utilise your Hadoop cluster.

Known Limitations and Issues

Currently, the only known limitations and issues are some extreme use cases which will not apply to most users. For example, Juju will not let users remove the Hadoop-Config relation, so switching between HDFS clusters is not currently possible.

If you find any further issues with this charm, please do let us know! You can find our contact details below.

Contact Information

Thank you for using the Druid Config charm - we hope you find it useful! For more information, here are our contact details:

Spicule: http://www.spicule.co.uk
Anssr: http://anssr.io
Email: info@spicule.co.uk

For assistance with Juju, please refer to the Juju Discourse.

Druid

For more information about Druid, please refer to the Druid website.