`SparkSubmitArguments` — spark-submit’s Command-Line Argument Parser

SparkSubmitArguments is a custom SparkSubmitArgumentsParser to handle the command-line arguments of spark-submit script that the actions (i.e. submit, kill and status) use for their execution (possibly with the explicit env environment).

Note	`SparkSubmitArguments` is created when launching `spark-submit` script with only `args` passed in and later used for printing the arguments in verbose mode.

Calculating Spark Properties — `loadEnvironmentArguments` internal method

loadEnvironmentArguments(): Unit

loadEnvironmentArguments calculates the Spark properties for the current execution of spark-submit.

loadEnvironmentArguments reads command-line options first followed by Spark properties and System’s environment variables.

Note	Spark config properties start with `spark.` prefix and can be set using `--conf [key=value]` command-line option.

`handle` Method

protected def handle(opt: String, value: String): Boolean

handle parses the input opt argument and returns true or throws an IllegalArgumentException when it finds an unknown opt.

handle sets the internal properties in the table Command-Line Options, Spark Properties and Environment Variables.

`mergeDefaultSparkProperties` Internal Method

mergeDefaultSparkProperties(): Unit

mergeDefaultSparkProperties merges Spark properties from the default Spark properties file, i.e. spark-defaults.conf with those specified through --conf command-line option.

SparkSubmitArguments