Charmed Spark K8s

Manage Charmed Spark Service Accounts

This is an introduction to the CLI interface for creating, managing and configuring Charmed Spark service accounts.

Note Charmed Spark service accounts are designed to work seamlessly with the Spark Integration Hub charm, allowing you to manage Spark configuration using Juju relations. You can find more information about how to use configuration hub here.

Create Service Account

Note: In case using another namespace than default, make sure that it already exists in Kubernetes.

$ kubectl create namespace demonamespace
namespace/demonamespace created

Now we can define a service account within the scope of this namespace. In case we may have a configuration file with default settings for the account, we could also include those when defining the account. The syntax of the config file is identical to Spark Configuration Properties. For example

$ echo "spark.kubernetes.deploy-mode=cluster" > /home/demouser/conf/spark-overrides.conf

The command below creates a service account associated with configuration properties provided either via property file or explicit configuration arguments. The flags --primary specifies that the newly created account will be the primary account to be used. (If another primary exists, the latter account primary flag will be set to false.)

spark-client.service-account-registry create --username demouser --namespace demonamespace  --primary --properties-file /home/demouser/conf/spark-overrides.conf  --conf spark.app.name=demo-spark-app-overrides

List all service accounts

The command below displays a list of the service accounts available, and whether they are primary account or service accounts

spark-client.service-account-registry list

Add more entries to Service Account Configuration

The command below will upsert into the existing configuration associated with the account.

spark-client.service-account-registry add-config --username demouser --namespace demonamespace  --properties-file /home/demouser/conf/spark-overrides.conf  --conf spark.app.name=demo-spark-app-overrides

Remove entries from Service Account Configuration

The command below will remove the specified keys from existing configuration associated with the account.

spark-client.service-account-registry remove-config --username demouser --namespace demonamespace  --conf conf.key1.to.remove --conf conf.key2.to.remove

Print configuration for a given Service Account

This command will print the configuration for a given service account.

spark-client.service-account-registry get-config --username demouser --namespace demonamespace 

Delete Service Account Configuration

This command will delete the configurations associated to a given service account.

spark-client.service-account-registry clear-config --username demouser --namespace demonamespace 

Inspect Primary Service Account

This command will allow the user to find out which is the primary account, together with related configuration settings.

spark-client.service-account-registry get-primary

Cleanup a Service Account

This command will delete the service account together with the other resources created, e.g. secrets, role, role-bindings, etc.

spark-client.service-account-registry delete --username demouser --namespace demonamespace 

---