Bigdata Dev Apache Spark

Channel Revision Published Runs on
latest/stable 83 18 Mar 2021
Ubuntu 16.04 Ubuntu 14.04
latest/edge 70 18 Mar 2021
Ubuntu 14.04
juju deploy bigdata-dev-apache-spark
Show information

Learn to deploy on juju >

Platform:

Ubuntu
16.04 14.04

Learn about configurations >

  • driver_memory | string

    Default: 1g

    Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).

  • executor_memory | string

    Default: 1g

    Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).

  • resources_mirror | string

    URL used to fetch resources (e.g., Hadoop binaries) instead of the location specified in resources.yaml.

  • spark_bench_enabled | boolean

    Default: True

    When set to 'true' (the default), this charm will download and install the SparkBench benchmark suite from the configured URLs. When set to 'false', SparkBench will be removed from the unit, though any data stored in hdfs:///user/ubuntu/spark-bench will be preserved.

  • spark_bench_ppc64le | string

    Default: https://s3.amazonaws.com/jujubigdata/ibm/noarch/spark-bench-2.0-20151214-ffb72f23.tgz#sha256=ffb72f233eaafccef4dda6d4516f23e043d1b14b9d63734211f4d1968db86a3c

    URL (including hash) of a ppc64le tarball of SparkBench. By default, this points to a pre-built SparkBench binary based on sources in the upstream repository. This option is only valid when 'spark_bench_enabled' is 'true'.

  • spark_bench_x86_64 | string

    Default: https://s3.amazonaws.com/jujubigdata/ibm/noarch/spark-bench-2.0-20151214-ffb72f23.tgz#sha256=ffb72f233eaafccef4dda6d4516f23e043d1b14b9d63734211f4d1968db86a3c

    URL (including hash) of an x86_64 tarball of SparkBench. By default, this points to a pre-built SparkBench binary based on sources in the upstream repository. This option is only valid when 'spark_bench_enabled' is 'true'.

  • spark_execution_mode | string

    Default: standalone

    Options are "local", "standalone", "yarn-client", and "yarn-cluster". Consult the readme for details on these options.

  • spark_version | string

    Default: 1.6.1-hadoop2.6.0

    Version of Apache Spark to be deployed.