--- title: Charmhub | Deploy Opensearch K8S using Charmhub - The Open Operator Collection description: Deploy the latest version of Opensearch K8S as a Kubernetes Operator on any cloud. url: https://charmhub.io/opensearch-k8s/docs/h-large-deployment --- # Opensearch K8S [Canonical](https://charmhub.io/publisher/data-platform "View all packages from Canonical") * [Canonical](https://charmhub.io/publisher/data-platform "View all packages from Canonical") Platform: 2/edge 22 ``` juju deploy opensearch-k8s --channel 2/edge ``` [Learn to deploy on juju >](https://juju.is/docs/juju/manage-applications) --- Share your thoughts on this charm with the community on discourse. [Join the discussion](https://discourse.charmhub.io/) # How to launch a large deployment The Charmed OpenSearch operator can be deployed at scale to support large deployments. This guide explains how to launch a large deployment of OpenSearch using Juju. ## [Summary](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#p-33802-summary) * [OpenSearch node roles](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#opensearch-node-roles) + [Set roles](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#set-roles) - [Auto-generated roles](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#auto-generated-roles) - [User set roles](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#user-set-roles) * [Deploy a large OpenSearch cluster](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#deploy-a-large-opensearch-cluster) + [Deploy the clusters](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#deploy-the-clusters) + [Add the required relations](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#add-the-required-relations) - [Configure TLS encryption](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#configure-tls-encryption) - Form the large cluster + [Form the OpenSearch cluster](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#form-the-opensearch-cluster-11) --- ## [OpenSearch node roles](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#p-33802-opensearch-node-roles) When deploying OpenSearch at scale, it is important to understand the `roles` that nodes can assume on a cluster. Amongst the [multiple roles](https://opensearch.org/docs/latest/tuning-your-cluster/) supported by OpenSearch, two notable roles are especially crucial for a successful cluster formation: * `cluster_manager`: assigned to nodes responsible for handling cluster-wide operations such as creating and deleting indices, managing shards, and rebalancing data across the cluster. Every cluster has a single `cluster_manager` node elected as the master node among the `cluster_manager` eligible nodes. * `data`: assigned to nodes which store and perform data-related operations like indexing and searching. Data nodes hold the shards that contain the indexed data. Data nodes can also be configured to perform ingest and transform operations. In charmed OpenSearch, data nodes can optionally be further classified into tiers - to allow for defining [index lifecycle management policies](https://opensearch.org/docs/latest/im-plugin/ism/index/): + `data.hot` + `data.warm` + `data.cold` There are also other roles that nodes can take on in an OpenSearch cluster, such as `ingest` nodes, and `coordinating` nodes etc. Roles in charmed OpenSearch are applied on the application level, in other words, all nodes get assigned the same set of roles defined for an application. ### [Set roles](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#p-33802-set-roles) Roles can either be set by the user or automatically generated by the charm. #### Auto-generated roles When no roles are set on the `roles` config option of the opensearch application, the charm automatically assigns the following roles to all nodes. ``` ["data", "ingest", "ml", "cluster_manager"] ``` #### User set roles There are currently two ways for users to set roles in an application: at deploy time, or via a config change. Note that a role change will effectively trigger a rolling restart of the OpenSearch application. To set roles at deploy time, run ``` juju deploy opensearch -n 3 --config roles="cluster_manager,data,ml" ``` To set roles later on through a config change, run ``` juju config opensearch roles="cluster_manager,data,ml" ``` > **Note:** We currently do not allow the removal of either `cluster_manager` or `data` roles. ## [Deploy a large OpenSearch cluster](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#p-33802-deploy-a-large-opensearch-cluster) The OpenSearch charm manages large deployments and diversity in the topology of its nodes through `juju integrations`. The cluster will consist of multiple integrated juju applications (clusters) with each application configured to have a mix of `cluster_manager` and `data` roles defined for its nodes. ### [Deploy the clusters](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#p-33802-deploy-the-clusters) **Caution**: Charmed OpenSearch supports performance profiles and will have different RAM consumption according to the profile chosen: * `production`: consumes 50% of the RAM available, up to 32G * `staging`: consumes 25% of the RAM available, up to 32G * `testing`: consumes 1G of RAM The configuration defaults to `production`, but for the examples below, testing will be chosen as it is assumed the deployment happens on a single LXD cluster. 1. First, deploy the orchestrator app. ``` juju deploy -n 3 \ opensearch main \ --config cluster_name="app" \ --channel 2/edge \ --config profile="testing" ``` As a reminder, since we did not set any role to this application, the operator will assign each node the `cluster_manager,coordinating_only,data,ingest,ml` roles. 2. (Optional, but recommended) Next, deploy a failover application with `cluster_manager` nodes to ensure high availability and fault tolerance. The failover app will take over the orchestration of the fleet in the events where the `main` app fails or gets removed. Thus, it is important that this application has the `cluster_manager` role as part of its roles to ensure the continuity of the existence of the cluster. ``` juju deploy -n 3 \ opensearch failover \ --config cluster_name="app" \ --config init_hold="true" \ --config roles="cluster_manager" \ --channel 2/edge \ --config profile="testing" ``` The failover nodes are not required for a basic deployment of OpenSearch. They are however highly recommended for production deployments to ensure high availability and fault tolerance. > **Note 1:** It is imperative that the `cluster_name` config values match between applications in large deployments. A cluster\_name mismatch will effectively prevent 2 applications from forming a cluster. > **Note 2:** It is imperative that only the main orchestrator app sets the `init_hold` config option to `false` (by default) - the non-main orchestrator apps should set the value to `true` to prevent the application from starting before being integrated with the main. 3. After deploying the nodes of the `main` app and additional `cluster_manager` nodes on the `failover`, we will deploy a new app with `data.hot` node roles. ``` juju deploy -n 3 \ opensearch data-hot \ --config cluster_name="app" \ --config roles="data.hot" \ --config init_hold="true" \ --channel 2/edge \ --config profile="testing" ``` 4. We also need to deploy a TLS operator to enable TLS encryption for the cluster. We will deploy the `self-signed-certificates` charm to provide self-signed certificates for the cluster. ``` juju deploy self-signed-certificates ``` 5. We can now track the progress of the deployment by running: ``` juju status --watch 1s ``` Once the deployment is complete, you should see the following output: ``` Model Controller Cloud/Region Version SLA Timestamp dev development localhost/localhost 3.5.3 unsupported 06:01:06Z App Version Status Scale Charm Channel Rev Exposed Message data-hot blocked 3 opensearch 2/edge 159 no Cannot start. Waiting for peer cluster relation... failover blocked 3 opensearch 2/edge 159 no Cannot start. Waiting for peer cluster relation... main blocked 3 opensearch 2/edge 159 no Missing TLS relation with this cluster. self-signed-certificates active 1 self-signed-certificates latest/stable 155 no Unit Workload Agent Machine Public address Ports Message data-hot/0 active idle 6 10.214.176.165 data-hot/1* active idle 7 10.214.176.7 data-hot/2 active idle 8 10.214.176.161 failover/0* active idle 3 10.214.176.194 failover/1 active idle 4 10.214.176.152 failover/2 active idle 5 10.214.176.221 main/0 blocked idle 0 10.214.176.231 Missing TLS relation with this cluster. main/1 blocked idle 1 10.214.176.57 Missing TLS relation with this cluster. main/2* blocked idle 2 10.214.176.140 Missing TLS relation with this cluster. self-signed-certificates/0* active idle 9 10.214.176.201 Machine State Address Inst id Base AZ Message 0 started 10.214.176.231 juju-d6b263-0 ubuntu@22.04 Running 1 started 10.214.176.57 juju-d6b263-1 ubuntu@22.04 Running 2 started 10.214.176.140 juju-d6b263-2 ubuntu@22.04 Running 3 started 10.214.176.194 juju-d6b263-3 ubuntu@22.04 Running 4 started 10.214.176.152 juju-d6b263-4 ubuntu@22.04 Running 5 started 10.214.176.221 juju-d6b263-5 ubuntu@22.04 Running 6 started 10.214.176.165 juju-d6b263-6 ubuntu@22.04 Running 7 started 10.214.176.7 juju-d6b263-7 ubuntu@22.04 Running 8 started 10.214.176.161 juju-d6b263-8 ubuntu@22.04 Running 9 started 10.214.176.201 juju-d6b263-9 ubuntu@22.04 Running ``` ### [Add the required relations](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#p-33802-add-the-required-relations) #### Configure TLS encryption The Charmed OpenSearch operator does not function without TLS enabled. To enable TLS, integrate the `self-signed-certificates` with all opensearch applications. ``` juju integrate self-signed-certificates main juju integrate self-signed-certificates failover juju integrate self-signed-certificates data-hot ``` Once the integrations are established, the `self-signed-certificates` charm will provide the required certificates for the OpenSearch clusters. Once TLS is fully configured in the `main` app, the latter will start immediately. As opposed to the other apps which are still waiting for the `admin` certificates to be shared with them by the `main` orchestrator. When the `main` app is ready, `juju status` will show something similar to the sample output below: ``` Model Controller Cloud/Region Version SLA Timestamp dev development localhost/localhost 3.5.3 unsupported 06:03:49Z App Version Status Scale Charm Channel Rev Exposed Message data-hot blocked 3 opensearch 2/edge 159 no Cannot start. Waiting for peer cluster relation... failover blocked 3 opensearch 2/edge 159 no Cannot start. Waiting for peer cluster relation... main active 3 opensearch 2/edge 159 no self-signed-certificates active 1 self-signed-certificates latest/stable 155 no Unit Workload Agent Machine Public address Ports Message data-hot/0 active idle 6 10.214.176.165 data-hot/1* active idle 7 10.214.176.7 data-hot/2 active idle 8 10.214.176.161 failover/0* active idle 3 10.214.176.194 failover/1 active idle 4 10.214.176.152 failover/2 active idle 5 10.214.176.221 main/0 active idle 0 10.214.176.231 9200/tcp main/1 active idle 1 10.214.176.57 9200/tcp main/2* active idle 2 10.214.176.140 9200/tcp self-signed-certificates/0* active idle 9 10.214.176.201 Machine State Address Inst id Base AZ Message 0 started 10.214.176.231 juju-d6b263-0 ubuntu@22.04 Running 1 started 10.214.176.57 juju-d6b263-1 ubuntu@22.04 Running 2 started 10.214.176.140 juju-d6b263-2 ubuntu@22.04 Running 3 started 10.214.176.194 juju-d6b263-3 ubuntu@22.04 Running 4 started 10.214.176.152 juju-d6b263-4 ubuntu@22.04 Running 5 started 10.214.176.221 juju-d6b263-5 ubuntu@22.04 Running 6 started 10.214.176.165 juju-d6b263-6 ubuntu@22.04 Running 7 started 10.214.176.7 juju-d6b263-7 ubuntu@22.04 Running 8 started 10.214.176.161 juju-d6b263-8 ubuntu@22.04 Running 9 started 10.214.176.201 juju-d6b263-9 ubuntu@22.04 Running ``` ### [Form the OpenSearch cluster](https://charmhub.io/opensearch-k8s/docs/h-large-deployment#p-33802-form-the-opensearch-cluster) Now, in order to form the large OpenSearch cluster (constituted of all the 3 previous opensearch apps), integrate the `main` charm to the `failover` and `data-hot` juju apps. ``` juju integrate main:peer-cluster-orchestrator failover:peer-cluster juju integrate main:peer-cluster-orchestrator data-hot:peer-cluster juju integrate failover:peer-cluster-orchestrator data-hot:peer-cluster ``` Once the relations are added, the `main` application will orchestrate the formation of the OpenSearch cluster. This will start the rest of the nodes in the cluster. You can track the progress of the cluster formation by running: ``` juju status --watch 1s ``` Once the cluster is formed and all nodes are up and ready, `juju status` will show something similar to the sample output below: ``` Model Controller Cloud/Region Version SLA Timestamp dev development localhost/localhost 3.5.3 unsupported 06:11:18Z App Version Status Scale Charm Channel Rev Exposed Message data-hot active 3 opensearch 2/edge 159 no failover active 3 opensearch 2/edge 159 no main active 3 opensearch 2/edge 159 no self-signed-certificates active 1 self-signed-certificates latest/stable 155 no Unit Workload Agent Machine Public address Ports Message data-hot/0 active idle 6 10.214.176.165 9200/tcp data-hot/1* active idle 7 10.214.176.7 9200/tcp data-hot/2 active idle 8 10.214.176.161 9200/tcp failover/0* active idle 3 10.214.176.194 9200/tcp failover/1 active idle 4 10.214.176.152 9200/tcp failover/2 active idle 5 10.214.176.221 9200/tcp main/0 active idle 0 10.214.176.231 9200/tcp main/1 active idle 1 10.214.176.57 9200/tcp main/2* active idle 2 10.214.176.140 9200/tcp self-signed-certificates/0* active idle 9 10.214.176.201 Machine State Address Inst id Base AZ Message 0 started 10.214.176.231 juju-d6b263-0 ubuntu@22.04 Running 1 started 10.214.176.57 juju-d6b263-1 ubuntu@22.04 Running 2 started 10.214.176.140 juju-d6b263-2 ubuntu@22.04 Running 3 started 10.214.176.194 juju-d6b263-3 ubuntu@22.04 Running 4 started 10.214.176.152 juju-d6b263-4 ubuntu@22.04 Running 5 started 10.214.176.221 juju-d6b263-5 ubuntu@22.04 Running 6 started 10.214.176.165 juju-d6b263-6 ubuntu@22.04 Running 7 started 10.214.176.7 juju-d6b263-7 ubuntu@22.04 Running 8 started 10.214.176.161 juju-d6b263-8 ubuntu@22.04 Running 9 started 10.214.176.201 juju-d6b263-9 ubuntu@22.04 Running ``` **Caution**: The cluster will not come online if no `data` nodes are available. Ensure the `data` nodes are deployed and ready before forming the cluster. **Reminder1**: In order to form a large deployment out of multiple juju apps, all applications must have the same `cluster_name` config option value or not set it at all, in which case it will be auto-generated in the main orchestrator and inherited by the other members. **Reminder2:** `init_hold` must be set to `true` for any subsequent (non main orchestrator) application. Otherwise the application may start and never be able to join the rest of the deployment fleet. ---