Spark

Channel Revision Published Runs on
latest/stable 73 17 Dec 2020
Ubuntu 16.04 Ubuntu 14.04
latest/stable 2 17 Dec 2020
Ubuntu 16.04 Ubuntu 14.04
latest/candidate 73 17 Dec 2020
Ubuntu 16.04
latest/beta 73 17 Dec 2020
Ubuntu 16.04
latest/edge 90 21 Oct 2021
Ubuntu 18.04 Ubuntu 16.04
latest/edge 72 17 Dec 2020
Ubuntu 18.04 Ubuntu 16.04
juju deploy spark
Show information

Learn to deploy on juju >

Platform:

Ubuntu
18.04 16.04 14.04

Learn about configurations >

  • bigtop_version | string

    Default: 1.2.1

    Apache Bigtop release version. The default, '1.2.1' will use the current GA release, Bigtop 1.2.1, for all hiera data, puppet recipes, and installable packages. Set this to 'master' to use the latest upstream bits.

  • cuda-version | string

    Default: 9.1.85-1

    Version of the cuda-repo deb to install. Valid options can be found at: http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64

  • driver_memory | string

    Default: 1g

    Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).

  • executor_memory | string

    Default: 1g

    Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).

  • install-cuda | boolean

    Default: True

    Install the CUDA binaries if capable hardware is present (True by default). Set to False to disable CUDA installation regardless of capable hardware.

  • spark_bench_enabled | boolean

    When set to 'true', this charm will download and install the Spark-Bench benchmark suite from the configured URLs. When set to 'false' (the default), Spark-Bench will not be installed on the unit, though any data stored in hdfs:///user/ubuntu/spark-bench from previous installations will be preserved. Note that Spark-Bench has not been verified to work with Spark 2.1.x.

  • spark_bench_url | string

    Default: https://s3.amazonaws.com/jujubigdata/ibm/noarch/SparkBench-2.0-20170403.tgz#sha256=709caec6667dd82e42de25eb8bcd5763ca894e99e5c83c97bdfcf62cb1aa00c8

    URL (including hash) of a Spark-Bench tarball. By default, this points to a pre-built Spark-Bench binary based on sources in the upstream repository. This option is only valid when 'spark_bench_enabled' is 'true'.

  • spark_execution_mode | string

    Default: standalone

    Options are "local", "standalone", "yarn-client", and "yarn-cluster". Consult the readme for details on these options.