Bigdata Dev Apache Spark
- Juju Big Data Development
- Big Data
Channel | Revision | Published | Runs on |
---|---|---|---|
latest/stable | 83 | 18 Mar 2021 | |
latest/edge | 70 | 18 Mar 2021 | |
latest/edge | 1 | 18 Mar 2021 |
juju deploy bigdata-dev-apache-spark
Deploy universal operators easily with Juju, the Universal Operator Lifecycle Manager.
Platform:
-
driver_memory | string
Default: 1g
Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).
-
executor_memory | string
Default: 1g
Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).
-
resources_mirror | string
URL used to fetch resources (e.g., Hadoop binaries) instead of the location specified in resources.yaml.
-
spark_bench_enabled | boolean
Default: True
When set to 'true' (the default), this charm will download and install the SparkBench benchmark suite from the configured URLs. When set to 'false', SparkBench will be removed from the unit, though any data stored in hdfs:///user/ubuntu/spark-bench will be preserved.
-
spark_bench_ppc64le | string
Default: https://s3.amazonaws.com/jujubigdata/ibm/noarch/spark-bench-2.0-20151214-ffb72f23.tgz#sha256=ffb72f233eaafccef4dda6d4516f23e043d1b14b9d63734211f4d1968db86a3c
URL (including hash) of a ppc64le tarball of SparkBench. By default, this points to a pre-built SparkBench binary based on sources in the upstream repository. This option is only valid when 'spark_bench_enabled' is 'true'.
-
spark_bench_x86_64 | string
Default: https://s3.amazonaws.com/jujubigdata/ibm/noarch/spark-bench-2.0-20151214-ffb72f23.tgz#sha256=ffb72f233eaafccef4dda6d4516f23e043d1b14b9d63734211f4d1968db86a3c
URL (including hash) of an x86_64 tarball of SparkBench. By default, this points to a pre-built SparkBench binary based on sources in the upstream repository. This option is only valid when 'spark_bench_enabled' is 'true'.
-
spark_execution_mode | string
Default: standalone
Options are "local", "standalone", "yarn-client", and "yarn-cluster". Consult the readme for details on these options.
-
spark_version | string
Default: 1.6.1-hadoop2.6.0
Version of Apache Spark to be deployed.