Spark

Learn about configurations >

• bigtop_version | string

  Default: 1.2.1

  Apache Bigtop release version. The default, '1.2.1' will use the current GA release, Bigtop 1.2.1, for all hiera data, puppet recipes, and installable packages. Set this to 'master' to use the latest upstream bits.

• cuda-version | string

  Default: 9.1.85-1

  Version of the cuda-repo deb to install. Valid options can be found at: http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64

• driver_memory | string

  Default: 1g

  Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).

• executor_memory | string

  Default: 1g

  Specify gigabytes (e.g. 1g) or megabytes (e.g. 1024m). If running in 'local' or 'standalone' mode, you may also specify a percentage of total system memory (e.g. 50%).

• install-cuda | boolean

  Default: True

  Install the CUDA binaries if capable hardware is present (True by default). Set to False to disable CUDA installation regardless of capable hardware.

• spark_bench_enabled | boolean

  When set to 'true', this charm will download and install the Spark-Bench benchmark suite from the configured URLs. When set to 'false' (the default), Spark-Bench will not be installed on the unit, though any data stored in hdfs:///user/ubuntu/spark-bench from previous installations will be preserved. Note that Spark-Bench has not been verified to work with Spark 2.1.x.

• spark_bench_url | string

  Default: https://s3.amazonaws.com/jujubigdata/ibm/noarch/SparkBench-2.0-20170403.tgz#sha256=709caec6667dd82e42de25eb8bcd5763ca894e99e5c83c97bdfcf62cb1aa00c8

  URL (including hash) of a Spark-Bench tarball. By default, this points to a pre-built Spark-Bench binary based on sources in the upstream repository. This option is only valid when 'spark_bench_enabled' is 'true'.

• spark_execution_mode | string

  Default: standalone

  Options are "local", "standalone", "yarn-client", and "yarn-cluster". Consult the readme for details on these options.